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1  Introduction 

The  objective  of  Information  Warfighter  Effectiveness  (IWE)  Delivery  Order  0006, 
Commander’s  Decision  Aid  for  Predicted  Battlespace  Awareness  (CDA4PBA),  was  to 
develop  and  demonstrate  human-centered  decision-making  technologies  to  improve 
processes,  performance,  tools,  and  training  to  support  a  commanders’  predictive  battle- 
space  awareness  ability. 

The  program  provided  an  understanding  of  the  decisions  and  other  cognitive  work 
associated  with  Predicted  Battlespace  Awareness  (PBA)  for  Joint  Force  and  Air  Force 
Commanders,  their  staff  and  intelligence  support  functions.  CDA4PBA  supports  a 
commander’s  PBA  by  proposing  algorithms  and  high-level  concept  visualizations 
supporting  these  algorithms.  This  allows  the  commander  to  identify  and  rehearse  actions 
to  counter  the  enemy’s  actions  before  the  enemy  acts.  This  program  addressed  these 
advanced  visualizations  and  work-centered  decision  aides. 
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2  Approach 

The  project  team  consisted  of  SRA  International,  Inc.  (SRA)  as  the  prime  contractor, 
Charles  River  Analytics,  DMM  Ventures,  and  ManTech  Aegis  as  subcontractors.  As 
prime  contractor,  SRA  International  provided  overall  project  direction,  participated  in 
subcontractor  activities,  and  managed  the  project.  Charles  River  Analytics  brought 
experience  through  a  subject-matter  expert  (SME)  who  participated  as  a  United  States  Air 
Force  (USAF)  Scientific  Advisory  Board  member  on  the  panel  for  ‘Predictive 
Battlespace  Awareness  to  Improve  Military  Effectiveness’  as  well  as  knowledge  in  expert 
systems  and  human  factors  engineering.  DMM  Ventures  brought  experience  through  an 
SME  who  served  as  a  USAF  Colonel  and  helped  advance  strategy  planning  and 
developed  effects-based  operations  doctrine.  ManTech  Aegis  brought  a  unique  cognitive 
task  analysis  methodology  for  capturing  user  goals,  processes,  and  information 
requirements  in  the  PBA  domain. 

The  project  plan  included  identifying  PBA  requirements  for  USAF  users  by  completing  a 
literature  review,  data  collections,  and  a  cognitive  task  analysis  (CTA).  Given  that  PBA 
was  a  relatively  new  concept,  substantiated  by  lack  of  doctrinal  definition,  the  first  step 
was  to  find  out  exactly  what  PBA  meant  to  the  users  within  the  USAF.  PBA  doctrine, 
instructions,  and  pamphlets  provided  limited  insight,  in  that  they  were  still  being 
formulated  during  the  project.  The  immature  and  unofficial  PBA  lexicon  led  to 
differences  in  how  individuals  and  organizations  interpreted  the  purpose,  approach,  and 
expectations  of  PBA.  Two  typical  questions  posed  early  in  the  knowledge  elicitation 
process  were  “What  is  PBA?”  and  “How  do  you  use  PBA?” 

Three  USAF  sites  representing  a  cross-section  of  “users  of  PBA”  were  selected  for  initial 
knowledge  elicitations.  Existing  operational  conflicts  posed  a  risk  that  knowledge 
elicitations  would  be  delayed  or  unavailable.  The  information  from  these  meetings  was 
necessary  to  build  a  network  of  additional  potential  site  visits.  Following  the  CTA,  the 
project  continued  with  defining  user  requirements,  from  which  visualization  concepts 
would  be  derived  and  developed. 
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2.1  Refine  Project  Plan 

The  project  was  funded  to  put  the  emphasis  on  data  collections  and  user  requirements. 
However,  the  first  planned  data  collection  was  delayed  several  months  due  to  scheduling 
conflicts  at  user  sites  brought  on  by  the  operational  realities  of  on-going  real-world 
conflicts.  The  last  originally  planned  data  collection  occurred  shortly  after  the  time  at 
which  user  requirements  were  to  be  developed.  The  CTA  process,  though,  required 
several  months  development  time  for  the  initial  report,  followed  by  additional  data 
collections  to  further  refine  the  findings  prior  to  final  analysis.  Therefore,  initial  user 
requirements  were  developed  in  parallel  with  the  CTA  report,  as  results  became 
available.  Furthermore,  initial  knowledge  elicitations  indicated  PBA  was  pervasive  and 
influenced  many  elements  within  the  Air  &  Space  Operations  Center  (AOC),  so 
developing  user  requirements  would  be  time-consuming.  The  team  was  looking  for  a 
focus  area  to  help  drive  PBA  and  give  the  project  momentum  to  move  forward  in  lieu  of 
immediately  available  user  requirements. 

2.2  Further  Refine  Project  Plan 

While  the  CTA  was  moving  at  a  slower  pace  than  expected,  the  pressure  to  develop  a 
product  led  to  reprioritization  and  development  in  a  parallel  path  with  the  CTA.  The 
project  team  had  significant  experience  with  predictive  algorithm  development  and  those 
capabilities  were  leveraged  to  complement  both  requirement  and  visualization 
development.  Two  predictive  algorithm  approaches  were  pursued:  one  very  basic  that 
involved  integrating  existing  capabilities  to  create  a  quick-turn  platform  for 
demonstrating  decision-aiding  and  visualization  concepts  via  existing  applications;  and 
another  higher  risk,  higher  return  approach  at  a  more  conceptual  level. 

These  approaches  were  prompted  by  the  team’s  desire  to  develop  a  CDA4PBA 
demonstration  capability  beyond  storyboard  concepts,  and  to  provide  a  robust 
demonstration  for  a  meeting  by  the  PBA  requirements  group  at  the  Pentagon  (initially 
required  at  the  three-letter  level,  XOI).  The  goal  was  to  spiral  quickly  through  these 
capabilities.  However,  funding  reductions  occurred  shortly  after  spiral  one  was  kicked  off 
and  subsequent  spirals  could  not  be  developed. 
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3  Project  Activities 

The  project  followed  a  human-centered,  systems  engineering  process.  The  project  began 
with  a  literature  review  and  preparations  for  data  collections  or  knowledge  elicitation 
sessions  at  user  sites,  to  define  PBA  and  its  use  in  the  USAF  AOC.  Findings  from  the 
knowledge  elicitations  were  used  to  generate  user-focused  system  requirements.  These 
requirements  were  used  to  derive  and  design  PBA  visualization  prototype  concepts. 
While  the  project  did  not  follow  this  plan  in  the  exact  serial  sequence,  for  reasons  stated 
earlier,  the  basic  knowledge  capture,  requirements  definition,  and  concept  development 
occurred,  but  often  in  parallel  and  with  a  priori  knowledge,  as  necessary. 

3.1  Understanding  PBA 

In  order  to  develop  visualizations,  the  PBA  environment  had  to  first  be  understood. 
Several  techniques  were  used  to  perform  analysis  of  the  information  collected,  to  include: 
work  flow,  performance,  process,  control  and  functional  requirements,  collaboration 
technologies,  information  flow,  decisions,  and  strategy  analysis  (in  the  context  of  work 
performed  by  JFACC,  as  it  applies  to  the  integration  of  PBA).  Special  focus  was  on 
cognitive  requirements,  perception  requirements,  comprehension  and  projection  (and 
their  impact  on  information  displays,  with  respect  to  ordering),  retrieval,  and  other 
aspects  that  may  impact  cognitive  capabilities. 

3.1.1  Document  Review 

A  scientific  and  technical  literature  review  was  conducted  to  determine  the  current  state 
of  the  art  in  the  PBA  domain.  A  repository  was  established  so  all  team  members  could 
review,  access,  and  build  upon  available  references.  References  used  in  developing  the 
WDAR  are  included  in  Section  8. 

3.1.2  Knowledge  Elicitations 

The  objective  of  the  knowledge  elicitations  (KE)  was  to  gather,  through  a  combination  of 
methods,  a  complementary  set  of  information  about  the  PBA  decision-making  problem 
space.  Typically,  the  work  domain  analysis  is  performed  based  on  a  variety  of  KE 
activities.  This  involves  interactions  with  expert  practitioners  in  the  domain  and  includes 
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face-to-face  interviews  with  the  experts,  watching  the  experts  work  in  the  domain,  verbal 
protocol  techniques,  and  other  Cognitive  Task  Analysis  (CTA)  and  Cognitive  Work 
Analysis  (CWA)  methods.  This  was  an  iterative,  progressively  deepening  process.  Each 
step  taken  expands  the  base  of  knowledge  -  providing  the  opportunity  to  take  the  next 
step.  Making  progress  on  one  line  of  inquiry  (understanding  one  aspect  of  the  field  of 
practice)  created  the  room  to  make  progress  on  another. 

The  initial  base  of  knowledge  regarding  PBA,  and  how  practitioners  function  within  it, 
was  limited.  A  number  of  KE  techniques  were  used  to  expand  on  and  enrich  the  base 
understanding  and  evolve  an  ACWA  model  from  which  ideas  for  improved  support  could 
be  generated.  The  selection  of  which  technique(s)  to  use  and  how  many  techniques  to 
employ  was  motivated  by  the  need  to  produce  a  model  of  the  field  of  practice  and  a 
model  of  how  domain  practitioners  operate  in  that  field.  The  modeling  process  generally 
requires  the  use  of  multiple  converging  techniques  that  include  techniques  that  focus  on 
understanding  the  domain  demands  as  well  as  techniques  that  focus  on  understanding  the 
knowledge  and  strategies  of  domain  practitioners. 

The  KE  process  is  highly  opportunistic.  The  particular  set  of  techniques  that  were 
selected  were  determined  strongly  by  the  pragmatics  of  the  specific  local  conditions. 
Typically,  access  to  domain  practitioners  is  very  limited.  In  these  cases,  other  sources  of 
domain  knowledge  (e.g.  written  documents)  were  maximally  exploited  before  relying  on 
domain  experts.  In  some  cases,  observing  domain  experts  in  actual  work  practice  (e.g., 
using  ethnographic  methods  or  simulator  studies)  may  be  impractical,  and  in  those  cases, 
using  structured  interview  techniques  may  be  the  most  practical  method  available.  In 
cases  where  domain  experts  were  not  accessible  at  all  (e.g.,  in  highly  classified 
government  applications),  looking  for  surrogate  experts  (e.g.,  individuals  who  have 
performed  the  task  in  the  past)  or  examining  analogous  domains  is  necessary. 

One  key  element  of  the  KE  is  that  it  is  performed  iteratively  with  the  ACWA  effort.  As 
interim  results  from  the  modeling  task  become  available,  they  were  used  as  material  for 
further  elicitation.  The  process  of  constructing  the  ACWA  artifacts  provided 
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requirements  for  the  information  needed  to  enrich  the  model.  As  an  additional  benefit,  the 
Functional  Abstraction  Network  (FAN)  model  has  been  shown  to  be  extremely  powerful 
in  integrating  seemingly  disparate  sources  of  information  from  a  KE  process  into  a 
unified  analysis  and  design  framework.  Thus,  the  focus  of  the  KE  task  was  to,  in  an 
iterative  and  participatory  manner,  provide  the  data  necessary  to  construct  the  set  of 
ACWA  artifacts.  These  artifacts  and  approaches  were  summarized  in  the  WDAR. 

The  KE  approach  in  this  project  specifically  utilized  a  combination  of  methods.  These 
included  the  following: 

•  Reviewing  relevant  technical  documentation  on  PBA; 

•  Discussing  with  SMEs  the  difficult  PBA  scenarios  and  how  they 
impact  the  decision-making  process; 

•  Discussing  with  SMEs  the  FAN  (and  associated  cognitive  and 
information  requirements)  and  its  gaps  in  modeling  the  work  domain; 
and 

•  Observing  simulation  and  training  exercises  related  to  PBA  in  order  to 
operationally  validate  (or  negate)  the  analytical  findings. 

Appendix  I  contains  the  basic  data  collection  plan  for  knowledge  elicitations  at  user  sites. 
The  data  collection  plan  leveraged  the  opportunistic  nature  of  knowledge  elicitations, 
thus  the  plan  was  not  a  rigid  structured  process,  but  rather  a  high-level  guide  from  which 
deviations  can  be  developed  and  pursued.  Table  1  summarizes  the  KE  details  including 
interview  sites,  number  of  interviews  and  interviewees,  date  which  the  interviews 
occurred,  and  the  perspective  user’s  brought  to  PBA.  The  initial,  planned  data  collections 
included  the  608*  AIS  (Barksdale  AFB)  for  a  training  and  operational  perspective;  the 
614*  SIS  (Vandenberg  AFB)  for  a  space  operations  perspective;  and  the  32"^  AIS 
(Ramstein  AFB)  for  a  EUCOM  perspective.  These  sites  were  selected  to  provide  a  cross- 
section  of  user  and  operational  perspectives  with  respect  to  PBA.  KE  interviews  spanned 
the  entire  project  and  provided  valuable  insight  with  respect  to  those  users.  Figure  1 
provides  a  timeline  for  KE  and  other  project  related  activities. 
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Table  1.  KE  site  visit  and  briefing  details  (bold  are  original  elicitation  sites) 


Site 

Number  of 
Interviews 

Number  of 
Interviewees 

Date 

User  Perspective 

AFC2ISR/INO 

1 

5 

Mar  ‘04 

PBA  Requirements  WG  - 
briefing 

JFACC  Mentors 

1 

2 

Jun  ‘04 

Commander  -  Senior 

Decision-Makers 

608***  AIS, 

Barksdale  AFB 

8 

15 

Aug  ‘04 

Operations  and  Training 

614*"  SIS, 

Vandenberg 

AFB 

8 

18 

Sep  ‘04 

Space  PBA 

AOC  Strategy 
Requirements 

WG 

6 

7 

Sep  ‘04 

Strategy-focused  PBA 

32"“  AIS, 

Ramstein  AFB 

4 

4 

Nov  ‘04 

Operational  in-theatre, 

Joint  and  EUCOM 

National  Air  and 
Spaee 

Intelligence 

Center  (NASIC) 

2 

7 

Mar  ‘05 

PBA  via  Users  who  issue 
responses  to  Information 
Requests 
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Figure  1.  Timeline  for  KE  site  visit  and  project  related  activities 
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The  team  understood  that  opportunistie  data  eolleetions  would  oeeur.  Those  colleetions 
were  equally  likely  early  in  the  projeet  as  late  in  the  projeet.  Furthermore,  the  dynamie 
nature  and  operational  realities  of  user  sites  meant  scheduling  and  executing  the 
knowledge  elicitations  would  be  challenging.  This  was  evidenced  by  several  delays 
which  pushed  the  planned  initial  data  collection  back  several  months. 

3.2  User  Requirements 

The  WDAR  was  focused  on  defining  how  PBA  is  used  in  the  AOC.  Defining  user 
requirements  via  the  WDAR  ensured  a  solid  starting  point  for  PBA  system  specifications, 
e.g.  information  which  the  user  needs  in  order  to  support  working  with  PBA.  These  user- 
focused  requirements  were  abstractions  of  cognitive  work  requirements  and  information 
relationship  requirements  identified  in  the  WDAR  and  they  served  as  a  foundation  for 
future  system  design. 

3.3  Prediction  Approaches 

Two  goals  were  established  in  refining  project  scope  discussed  in  Section  2.2.  The  first 
goal  was  to  obtain  a  clear  understanding  of  the  PBA  domain.  The  second  goal  was  to 
incorporate  that  knowledge  into  alternative  “prediction  model”  concepts  by  leveraging 
team  capabilities.  Two  baseline  methods  were  pursued  from  which  users  could  evaluate 
effectiveness  and  capabilities.  These  methods  would  serve  as  building  blocks  for  future 
system  enhancement  and  development. 

Data  collection  delays,  and  the  time  required  to  process  the  data  for  the  WDAR,  meant 
the  prediction  methods  had  to  be  developed  in  parallel  with  requirements  in  order  to  meet 
briefing  deadlines  in  which  CDA4PBA  would  be  compared  to  other  more  mature  PBA- 
related  programs. 

The  two  alternative  approaches  rely  on  system  models  to  make  predictions  about  future 
events.  One  method,  the  Forecast  Model  (FM)  is  a  collection  of  commercial  off-the-shelf 
(COTS)  and  government  off-the-shelf  (GOTS)  programs  integrated  through  a  single  to-be 
technology.  The  alternate  method,  a  Bayesian  Belief  Network  (BNet)  simulation, 
integrated  a  Bayesian  Belief  Network  COTS  program  with  a  DARPA-developed 
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simulation  to  provide  BNet  estimations  through  an  existing  visualization  framework 
environment. 

3.3.1  Forecast  Model  Concept 

The  Forecast  Model  is  one  of  two  approaches  investigated  as  a  prediction  methodology. 
The  Forecast  Model  was  proposed  early  in  refining  project  scope  and  the  concept  was 
considered  classic  high-risk,  high-reward.  The  concept  is  developed  around  COTS  and 
GOTS  technologies  with  the  initial  spiral  establishing  the  underlying  foundation. 

Fundamentally,  the  Forecast  Model  searches  analyst-specified  intelligence  sources  for 
key  words  and  maps  that  information  into  a  knowledge  base  available  to  instantiate 
system  models  of  interest.  The  technique  comprises  several  technologies.  The  front-end 
includes  an  intelligence  data  processor  which  parses  and  stores  word  phrases  in  a 
knowledge  base.  These  information  elements,  often  from  disparate  sources,  are  related 
through  a  semantic  reasoning  engine  so  that  items  of  specific  interest,  as  well  as 
“nuggets”  of  information  deemed  relevant,  are  brought  to  the  analyst’s  attention. 

A  user  at  one  operational  site  commented,  “We  spend  an  enormous  amount  of  time  trying 
to  collate  information  from  many  different  sources.  In  the  end,  we  don’t  know  whether  all 
the  appropriate  (available)  information  sources  were  included.  A  system  should  bring  us 
relevant  information  and  we  can  then  spend  time  conducting  the  analysis.” 

Further,  relevant  findings  related  in  time  and  space  are  captured  and  presented  to  the 
analyst  for  possible  action.  These  and  the  aforementioned  findings  may  have  been 
identified  by  the  analyst,  however,  the  Forecast  Model  leverages  a  computer’s  ability  to 
quickly  search  and  analyze  data  and  compare  many  different  sources  for  relevant 
information,  a  task  not  easily  performed  by  analysts. 

The  prediction  capability  comes  into  play  as  knowledge  base  elements  are  instantiated 
against  existing  system  models.  In  cases  where  a  model  does  not  exist,  one  can  be  created 
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and  iteratively  refined  by  parsing  additional  intelligenee  sourees.  After  the  model  or 
models  have  been  instantiated  to  the  analyst’s  satisfaetion,  an  estimate  can  be  obtained. 


The  Forecast  Model’s  high  risk,  high  reward  approach  was  pursued  for  several  reasons. 
First,  it  supports  Intelligence  Preparation  of  the  Battlespace  (IPB)  through  analysis  of  the 
adversary,  target  system  models,  and  enemy  Courses  of  Action  (COA).  Second,  a 
“forecasting”  element  is  provided  which  provides  the  magnitude  and  range  of  error  for 
estimations.  Third,  Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  management  is 
supported,  in  that  the  Forecast  Model  can  identify  information  which  is  unknown. 

Finally,  the  Forecast  Model  can  be  used  to  forecast  the  impact  of  friendly  actions.  The 
Forecast  Model  was  an  attractive  prediction  approach,  because  it  would  utilize  tools  and 
techniques  that  are  currently  available,  although  the  process  and  effort  required  to 
combine  these  tools  was  yet  to  be  determined. 

3.3.2  Bayesian  Belief  Network  Simulation  Model 

The  Bayesian  Belief  Network  (BNet)  simulation  is  the  second  of  two  approaches 
investigated  as  a  prediction  methodology.  The  BNet  concept  is  developed  around  COTS 
and  Defense  Advanced  Research  Projects  Agency  (DARPA)  technologies  and  was 
proposed  shortly  after  the  Forecast  Model  as  a  quick-turn  method  for  demonstrating 
predictive  concepts  via  static  causal  analysis.  The  spiral  one  BNet  approach  provided  a 
basic  status  capability  that  could  serve  as  a  test  bed  to  demonstrate  prediction  capabilities 
and  results  through  visualizations.  This  lower-risk  approach  yielded  a  foundation  from 
which  more  advanced  BNet-based  prediction  methodologies,  such  as  temporal  or 
dynamic  belief  networks,  could  be  developed. 

BNet  was  a  powerful  approach  with  respect  to  presenting  the  user  decisional  information. 
One  challenge,  however,  is  the  technique  requires  an  inordinate  amount  of  subject-matter 
expertise  in  order  to  build  the  underlying  models  which  drive  the  user  information 
display.  Considerable  effort  and  revolutionary  steps  would  potentially  be  required  to 
simplify  user  interactions  sufficiently  to  enable  a  typical  user  to  identify,  interpret,  and 
input  model  data  as  well  as  build  and  maintain  a  model  in  the  BNet  environment.  The 
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methodology  requires  updating  the  model  with  new  information,  when  available,  in  order 
to  improve  predictions. 

3.4  Smart  Request  For  Information 

The  importance  of  information  updates  to  the  models  became  very  apparent  while 
developing  the  prediction  approaches.  Updating  models  requires  understanding  the 
environment.  One  facet  of  information  updates  that  was  reported  frequently  in  the 
operational  community  was  the  “request  for  information”  or  RFl.  An  RFl  may  be  issued 
when  an  operational  question  must  be  answered.  A  recurring  theme  from  knowledge 
elicitations  at  operational  sites  was  the  ineffectiveness  associated  with  executing  an  RFl, 
including  both  the  method  for  issuing  and  the  response.  Two  themes  were  developed. 

First  was  a  solution  to  short-term  RFls  or  those  requiring  a  quick  response.  These  RFls 
appeared  to  benefit  most  from  a  structured  format  for  issuing  and  responding  to  RFls. 

This  approach  would  help  the  analyst  to  ask  a  better  question  and  ensure  the  information 
is  in  a  format  which  minimizes  follow-up  questions  from  a  superior  or  responding 
organization.  Second,  a  complementary  procedure  was  identified  which  supports  longer- 
term  RFls  and  the  associated  analysis  process.  This  would,  in  part,  be  developed  with 
knowledge  management  through  ontology  updates. 
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4  Findings  and  Results 

Findings  and  products  from  each  of  the  tasks  described  in  Section  3  are  described  below 
with  references  to  specific  documents  which  contain  the  full  analysis  or  documentation. 
These  results  helped  form  an  understanding  of  the  PBA  operational  environment. 

4.1  Work  Domain  Analysis  Report  (WDAR) 

The  Work  Domain  Analysis  Report  (WDAR)  presents  and  describes  the  results  of 
Applied  Cognitive  Work  Analysis  (ACWA)  of  the  Air  &  Space  Command  and  Control 
work  domain,  focusing  specifically  on  the  functions  and  cognitive  work  that  are 
accomplished  within  the  functional  scope  of  PBA.  The  state  of  the  analytic  artifacts  was 
described  in  order  to  convey  the  structure  of  the  analysis  output  and  significant  insights 
that  were  garnered.  The  WDAR  for  this  project  was  focused  on  PBA,  and  as  such, 
provided  an  extension  of  an  existing  WDAR  encompassing  Aerospace  Command  & 
Control.  The  document  is  located  in  ‘ CD A4PBA_WDAR_vl. 4.pdf . 

The  scope  of  the  WDAR  covers  the  cognitive  work  that  is  related  to  PBA,  with  special 
care  to  point  out  the  cognitive  work  that  is  specific  to  responsible  agents,  such  as  the 
commander,  as  well  as  the  cognitive  work  done  by  groups  under  their  command, 
supporting  their  decisions,  such  as  the  intelligence  group. 

The  WDAR  occurred  in  two  phases.  The  first  phase  which  occurred  during  the  first  half 
of  the  project,  produced  an  interim  report  and  analysis  focused  on  establishing  a  breadth 
of  coverage,  gaining  a  wide  level  of  understanding  across  the  domain.  The  WDAR 
interim  report,  however,  lacked  sufficient  processing  to  yield  the  critical  “user 
information  requirements”  necessary  to  make  explicit  decisions  about  what  information 
should  or  should  not  be  included  in  the  methods,  decision-support  systems,  and 
visualizations.  The  second  phase  allowed  for  a  depth  of  coverage,  which  identified 
detailed  cognitive  work  requirements  and  information  relationships  requirements  related 
to  PBA  within  various  functions  of  the  domain. 
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The  WDAR  contains  detailed  descriptions  of  the  cognitive  analysis  generated  using  the 
ACWA  process.  The  results  of  the  modeling  effort  are  discussed  in  terms  of  the  twelve 
Goal-Process  Nodes  (GPNs)  of  the  domain  within  the  report’s  scope,  listed  as  follows: 
“Successfully  Manage  Air  &  Space  Objectives,”  “Successfully  Manage  Air  &  Space 
Effects,”  “Successfully  Manage  Air  &  Space  Aims,”  “Successfully  Infer 
Adversary/Others’  Goals,”  “Successfully  Infer  Adversary/Others’  Will,”  “Successfully 
Posit  Adversary/Others’  Capabilities,”  “Successfully  Assess  Adversary/Others’ 
Behavior,”  “Successfully  Manage  Indicators,”  “Successfully  Provide  Intelligence 
Results,”  “Successfully  Provide  Applied  Collection  Power  Findings,”  “Successfully 
Manage  Attributes  of  Desired  Subjects,”  and  “Successfully  Maintain  the  Inventory  of 
“Subjects”  (Targets)  in  the  World.”  These  nodes  and  the  accompanying  PBA  scope  are 
shown  as  part  of  the  AC2  Functional  Abstraction  Network  (FAN)  in  Figure  2. 
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Figure  2.  Twelve  PBA  Goal-Process  Nodes  in  Air  &  Space  Command  and  Control 


13 


The  WDAR  is  divided  into  four  seetions: 

Section  A  -  Document  description  and  project  management  material. 

Section  B  -  A  brief  tutorial  for  how  to  read  and  understand  the  contents  of  the  WDAR.  It 
starts  with  an  overview  of  the  ACWA  methodology,  and  then  continues  to  a  high-level 
explanation  of  the  artifacts  of  ACWA,  which  serve  as  the  basis  of  analyses  presented  in 
Section  C. 

Section  C  -  This  section  comprises  the  majority  of  the  document  and  has  two  parts.  The 
first  part  includes  a  general  overview  of  the  Functional  Abstraction  Network  developed 
for  the  Strategy  Division’s  work  domain.  It  contains  a  brief  explanation  of  the  FAN  and 
includes  a  “cartoon”  and  detailed  version  of  the  FAN,  as  well  as  an  introduction  to  the 
four  elemental  scopes  of  PBA.  The  second  part  of  this  section  is  a  node-by-node 
description  of  the  in-scope  GPNs  that  comprise  the  FAN.  This  part  includes  “Overview,” 
“Goal,”  “Commodity,”  and  “Process”  sub-sections.  Cognitive  Work  Requirements 
(CWR)  and  Information  Relationship  Requirements  (IRR)  tables  follow  up  the 
description  of  the  GPN,  followed  by  a  section  to  highlight  the  GPN’s  relationships  to  the 
concept  of  PBA. 

Section  D  -  A  summary  of  the  analytic  effort  for  the  CDA4PBA  project  is  provided  in 
this  section. 

Major  findings  from  the  WDAR  can  be  summarized  as  follows: 

“PBA  is  everywhere” 

•  The  individual  scopes  of  PBA  (divided  by  doctrinally  defined  elements)  could 
have  been  more  encompassing  by  including  a  majority  of  the  goal-process  nodes 
for  all  sub  scopes.  However,  the  scopes  were  limited  in  their  coverage  in  an 
attempt  to  get  some  functional  traction  within  the  PBA  world.  Just  because  there 
was  a  brief  mention  of  a  concept  being  concerned  with  a  certain  element  of 
information  or  certain  process,  it  was  not  automatically  included  in  scope. 

•  Seeing  all  four  PBA  scopes  on  the  AC2  FAN  at  once,  gave  the  impression  that 
PBA  was  to  cover  ‘everything.’  There  is  a  desire  to  give  the  Commander  the 
ability  to  monitor  a  large  majority  of  cognitive  work  within  the  work  domain,  to 
provide  the  ‘awareness’  part  of  the  PBA  to  them.  However,  to  take  this  thin 
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amount  of  cognitive  work  (status  monitoring)  out  of  its  functional  context  is  to 
create  more  cognitive  work  because  of  the  subjective  division  of  a  goal  process 
node. 

•  There  is  no  magie  bullet  for  PBA;  the  eoverage  is  so  broad  and  eneapsulates  so 
many  necessary  deeisions,  that  there  is  not  a  single  way  to  support,  let  alone  do 
PBA.  It  requires  the  eoordination  of  many  sets  of  deeisions  and  deeision  makers 
to  provide  the  results  discussed  with  PBA. 

Tighter  Intelligence  Coupling 

•  In  the  Interim  WDAR,  there  was  an  interestingly  large  overlap  of  all  scopes  in  the 
Intelligenee  Analysis  region  of  the  FAN  (GPN  4.4  -  Indicators  and  GPN  6.3  - 
Intelligence  Results).  The  coding  of  these  scopes  was  changed  for  this  Updated 
WDAR,  because  in  fact  these  functional  processes  do  not  belong  in  the  seopes  of 
PBA,  whereas  they  are  the  KEY  funetional  support  to  eaeh  goal-proeess  that  is 
associated  with  PBA.  They  were  so  important;  that  they  were  not  decoupled  until 
a  further  detailed  inspeetion  took  plaee  in  the  seeond  half  of  the  analysis  period. 

•  This  stresses  the  importanee  of  tying  cognitive  work  within  the  intelligenee 
analysis  goal  proeesses  to  the  eonceptual  PBA  elements  via  the  speeific  support- 
supported  links  between  the  GPNs  in  the  FAN.  The  eloser  the  relationship 
between  Intelligenee  work  and  PBA  work,  the  more  accurate  PBA  predietions 
will  be. 

Status  Understanding 

•  The  Assessment  Seope  is  found  only  on  the  second  portion  of  several  goal- 
proeesses  within  the  domain.  The  current  state  of  Blue  (and  other  friendly) 
Forces,  the  current  state  of  Adversary  Forces  and  the  current  state  of  Gray  Forces 
is  information  that  would  be  desired  by  the  Commander. 

•  The  eognitive  work  that  is  required  to  understand,  monitor  and  define  the  status  of 
these  various  forees  is  a  culmination  point  of  decisions  throughout  many  goals 
within  the  domain.  The  Commander  may  only  be  eoneemed  about  making 
decisions  with  the  highest-level  status  they  received  to  support  the  PBA  seope  of 
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Assessment.  Nevertheless,  countless  other  warfighters  are  providing  the  grist  for 
the  mill  by  doing  the  low-level  decisions  within  the  Air  Force. 

Cognitive  Work  Requirements  -  Responsible  Agents 

The  focus  of  this  project  was  to  develop  decision  aids  for  the  Commander,  in  support  of 
PBA.  This  focus,  incorporated  with  the  understanding/insights  above,  meant  three  groups 
of  ‘Responsible  Agents’  were  defined:  Commander,  Support  and  Intelligence,  as  a  way 
to  discern  between  the  numerous  CWRs  and  IRRs  that  were  included  in  the  PBA  element 
scopes.  An  explanation  of  the  three  types  follows. 

The  four  PBA  Scopes  -  Intelligent  Preparation  of  the  Battlespace,  Target  Development, 
ISR  Strategy  and  Employment,  and  Assessment  are  each  illustrated  as  an  iceberg.  The 
metaphor  of  the  iceberg  is  used  to  illustrate  that  in  actuality  only  a  small  portion  of  the 
total  cognitive  work  related  to  each  PBA  Scope  is  found  “above  water,”  in  this  case  the 
portion  above  the  water  represents  the  decisions  that  are  being  made  by  the  Commander. 
The  Commander’s  decisions,  in  general,  are  of  the  highest  understanding  and  coverage  - 
for  example,  the  Commander  would  like  to  know  the  status  of  all  of  Blue  Forces’  desired 
effects,  to  get  an  understanding  of  the  current  state  of  effort.  The  commander  would  not 
want  to  hear  details  of  how  these  assessments  were  derived. 

The  Commander  tag  is  used  when  the  Commander  themselves  or  the  Commanders  staff 
is  the  responsible  agent.  Nevertheless,  for  the  Commander  to  have  this  overall 
understanding  of  the  effort,  a  culmination  of  various  other  decisions  and  information 
must  support  the  Commander’s  decisions  -  the  Support.  To  continue  with  the  metaphor, 
an  iceberg,  without  the  underlying  (underwater)  support  structure,  would  not  be  able  to 
break  the  water’s  surface.  Thus,  without  the  Support  cognitive  work  within  each  of  the 
PBA  scopes,  the  Commander’s  decisions  would  not  be  possible.  An  attempt  to  only 
support  the  Commander’s  decisions,  without  the  context  and  support  structure  of  the 
underlying  cognitive  work  requirements  would  be  inadequate  support  of  PBA. 

The  third  responsible  agent  for  CWRs  related  to  PBA  is  Intel.  The  CWRs  and  IRRs 
related  to  the  intelligence  process  are  not  directly  included  in  the  scope  of  PBA,  but  are 
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significant  in  the  successful  execution  of  the  PBA  related  eognitive  work.  The  closer  the 
relationship  between  the  eognitive  work  within  the  Intelligenee  analysis  and  the  cognitive 
work  within  the  PBA  scopes,  the  higher  fidelity  the  PBA  deeisions. 

The  ACWA  based  analysis  of  eognitive  work  assoeiated  with  PBA  in  the  AC2  domain 
captured  some  significant  insights  into  features  neeessary  to  support  PBA  in  an  AOC  of 
the  future.  The  proper  design  with  the  cognitive  work  identified  and  supported  to 
sueeessfiilly  manage  PBA  will  reduee  the  eognitive  workload  on  the  warfighter,  not 
create  a  larger  amount,  as  the  warfighter  is  required  to  adjust  to  a  deeision  support 
system. 

Information  from  the  May  and  August  2005  data  colleetions  could  not  be  incorporated 
into  the  WDAR  due  to  the  substantial  re-processing  that  was  required.  These  data  were 
noted  by  the  project  team  and  captured  in  a  trip  report. 

4.2  User  Requirements 

One  hundred  and  thirty-two  high-level,  user-focused  PBA  information  and  decision 
strategy  requirements  supporting  the  commander,  analysts,  and  intelligenee  staff  were 
identified  and  are  listed  by  WDAR  goal  process  node  (GPN)  in  Appendix  III.  Again,  the 
GPNs  represent  major  deeisions  whieh  occur  in  a  PBA  environment.  Note  these  are 
initial  user  information  requirements  based  on  the  Updated  WDAR  and  should  be 
validated  by  the  user  eommunity  prior  to  ineorporating  into  a  system  design. 

4.3  Forecast  Model 

The  Forecast  Model  determines  the  likelihood  of  future  model-based  aetivities  by 
searehing  available  intelligence  sources  and  instantiating  that  information  in  models. 
Another  powerful  feature  is  the  analyst’s  ability  to  off-load  time-consuming  data-mining 
activities  to  the  Forecast  Model,  leaving  more  time  for  the  analyst  to  interpret  findings 
discovered  from  these  potentially  disparate  intelligenee  sourees. 

The  Forecast  Model  begins  with  analysis  of  struetured  and  unstruetured  text  documents. 
Analysis  is  performed  on  words  and  phrases  to  extraet  meaning,  eontext  (structure),  and 
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relationships  (cause-effect  through  unknown).  The  mathematical  models  supporting 
analysis  algorithms  exist  and  some  of  the  technologies  exist.  The  spiral  one  approach  was 
to  develop  the  model  specification  within  Enterprise  Architect  5.0.  Enterprise  Architect 
enables  one  to  develop  Unified  Modeling  Language  (UML)  diagrams.  UML  is  an 
industry- standard  for  specifying,  visualizing,  constructing,  and  documenting  the  artifacts 
of  software  systems.  The  UML  Activity  Diagram  details  the  Forecast  Model  components, 
information  flows,  decisions,  and  requests  for  additional  information.  The  entire  model 
specification  within  Enterprise  Architect  is  provided  in  a  separate  file  (see  FM  Activity 
Diagram.rtf). 

Although  the  model  specification  was  described  through  activity  and  sequence  diagrams, 
the  process  is  complex,  and  as  such  required  a  thorough  user-centered  interface  design. 
Visualization  concepts  were  storyboarded  to  support  and  better  explain  user  interactions 
with  the  Forecast  Model.  Capabilities  specific  to  a  user-centered  design  for  non-technical 
users  included  1)  hiding  the  technical  details  of  the  process  (transparency),  2)  exposing 
additional  detail  upon  a  user’s  request  (drill-down),  and  3)  clearly  stating  the  rationale  for 
determining  outcomes.  Achieving  these  three  objectives  is  a  step  toward  building  a  user’s 
trust  in  operating  a  system  which  has  significant  and  complex  underlying  mechanics.  The 
Forecast  Model  was  also  a  tool  which  could  be  used  by  the  commander  as  well  as  his  or 
her  staff.  Thus,  views  were  tailored  to  support  the  information  requirements  for  these  two 
different  user  groups. 
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4.3.1  Activity  Diagram 

The  Activity  Diagram  is  shown  in  Figure  3  and  represents  a  high-level  view  of  system 
components  and  user  interactions.  This  diagram  served  as  a  foundation  for  the  type  and 
order  of  user  activities  as  well  as  system  activities. 
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Figure  3.  Forecast  Model  Activity  Diagram 
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4.3.2  Use  Cases 


The  next  step  following  Model  Specification  was  to  develop  use  cases  -  the  foundation 
for  user  interaction  and  a  building  block  for  developing  software  systems  and  prototypes. 
The  use  cases  were  kept  at  a  high  level  since  user  information  requirements  had  not  yet 
been  finalized.  The  use  case  descriptions,  provided  in  Appendix  II,  were  then  used  to 
generate  storyboards  and  visualizations  for  several  points  in  the  Forecast  Model  user 
interaction  process.  The  complete  high-level  use  cases  are  contained  in  ‘SRA  User 
Interactions  13Jun05  v3.doc’. 

4.3.3  Storyboard 

Significant  effort  was  involved  in  attempting  to  understand  user  interactions  within  the 
Forecast  Model.  While  the  use  cases  provided  a  high-level  understanding  of  the  Forecast 
Model,  additional  detail  was  required  to  complete  concept  visualizations.  This  detail  was 
captured  through  a  spiral  interview  process  with  the  Forecast  Model  developer.  The 
Forecast  Model,  however,  was  an  evolving  concept,  and  as  such,  user  interactions  and 
information  requirements  changed  frequently. 

Concept  visualizations  were  particularly  challenging,  because  the  Forecast  Model  is 
based  on  a  network  of  complex  mathematical  concepts  and  algorithms.  A  good  user- 
centered  design  requires  the  user  to  access  to  these  algorithms,  when  desired,  and  provide 
insight  into  how  results  are  formulated.  An  analyst  focused  visualization  concept  for 
management  of  system  models  is  shown  in  Figure  4.  In  this  concept,  the  analyst  has 
access  to  and  manages  relevant  system  models.  Management  includes  updates  or  edits,  as 
required,  through  inputs  to  both  text  and  graphical  formats.  The  analyst  also  has  access  to 
several  views  based  on  the  type  of  analysis  being  conducted,  e.g.  correlating  sources  or 
determining  the  goodness  of  fit.  Additional  functions  are  available  to  the  analyst  within 
the  Forecast  Model  environment  including  manipulating  graphical  models,  retrieving  a 
report,  and  issuing  a  Request  For  Information  (RFI).  (See  ‘FM  new  concept  vl.5.2.ppt’ 
for  a  storyboard  of  the  FM  process.) 
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Figure  4.  Analyst  focused  visualization  concept  for  management  of  system  models 
within  the  Forecast  Model 

4.4  Bayesian  Belief  Network  Simulation 

The  goal  for  the  spiral  I  prototype  software  was  to  demonstrate  visualization  concepts 
and  provide  a  foundation  from  which  Bayesian  Belief  Network  prediction  algorithms 
could  be  developed.  Future  spirals  would  extend  the  “static”  Bayesian  Network 
framework  to  temporal  or  dynamic  models  and  thus  provide  a  richer  environment  from 
which  to  build  predictive  capabilities.  The  prototype  software  used  the  Bayesian  Belief 
Network  engines  within  BNet  Builder  (COTS)  overlayed  with  visualization  concepts 
from  a  DARPA  simulation  effort  (Context-driven  InfoSpace  Configuration  for 
Augmented  Cognitive  Readiness  -  CIGAR). 
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The  prototype  served  two  purposes.  First,  it  provided  a  means  to  demonstrate  PBA 
visualization  eoncepts  tied  directly  to  results  generated  through  a  Bayesian  network 
(prediction)  engine.  Second,  the  prototype  provided  a  means  for  a  “quick-turn” 
demonstration  to  support  a  briefing  at  the  Pentagon  in  which  other  more  mature  projects 
would  be  demonstrating  the  results  of  their  efforts. 

4.5  Smart  Request  For  Information 

The  Smart  Request  For  Information  (SmartRFI)  concept  resulted  from  the  need  to 
support  information  exchanges  and  updates  to  both  prediction  model  concepts,  and  the 
ISR  system  in  general.  Initial  design  concepts  focused  on  identifying  the  essential 
elements  of  information  (EEI)  necessary  to  capture  the  requestor’s  intent.  These  EEIs 
formed  an  element  of  context.  To  quickly  arrive  at  context,  a  structured  RFI  template  was 
designed  to  help  the  requestor  better  focus  the  RFI  and  the  responder  better  focus  the 
effort  (see  Figure  5).  Supporting  context  was  an  environment  helping  the  requestor  and 
responder  conduct  research  with  emphasis  on  searching  related  RFIs,  a  SME  database, 
and  building  an  Ontology  or  knowledge  base  from  RFI  exchanges. 
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Figure  5.  Initial  conceptual  design  for  standardizing  an  RFI  and  context 
4.5.1  Design  Concept  Document 

The  SmartRFI  Design  Concept  Document  proposes  an  approach  for  implementing 
SmartRFI.  The  document  is  described  as  an  enhancement  to  Coliseum  in  which 
additional  functionality  is  considered,  such  as  Tracking,  Last  Time  Information  Is  of 
Value,  Product  Formats,  building  a  knowledge  base,  and  interfacing  with  Ontologies.  The 
document  does  not  consider  interactions  or  communication  with  existing  systems. 
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5  Findings 

The  CDA4PBA  project  resulted  in  several  findings.  First,  a  cognitive  analysis  was 
conducted  across  AOCs  with  varying  missions;  this  yielded  important  information 
regarding  prediction  and  PBA  for  commanders,  intelligence  staff,  and  support  staff  The 
analysis,  however,  was  focused  on  the  Air  Intelligence  Squadron  (AIS)  or  Space 
Intelligence  Squadron  (SIS)  within  each  AOC.  While  these  squadrons  comprise  the  main 
users  of  PBA,  and  provided  a  good  starting  point  for  understanding  PBA,  other  important 
users  and  consumers  exist. 

Second,  information  from  interviews  with  SMEs,  and  the  subsequent  cognitive  analysis, 
was  used  to  generate  an  initial  set  of  user-focused  system  and  information  requirements 
necessary  to  support  PBA  in  an  AOC  environment.  These  requirements  support  the 
commander  and  his  or  her  staffs  main  goals,  with  respect  to  using  PBA  information,  and 
provide  a  framework  from  which  a  preliminary  system  design,  including  user 
interactions,  can  be  developed. 

Third,  alternative  (and  potentially  complementary)  prediction  approaches  were  developed 
in  parallel  with  the  lengthy  and  complex  cognitive  analysis.  A  spiral  development  plan 
was  used  for  the  two  prediction  approaches.  However,  funding  limited  this  effort  to  one 
spiral. 

The  Forecast  Model  approach  provided  a  foundation  for  the  analysis  of  information 
leading  to  predictive  capability.  The  technologies  did  not  all  exist  and  thus  the  concept 
was  exploratory.  User  interactions  and  visualization  concepts  were  also  developed  to 
expose  the  underlying  capabilities. 

The  BNet  simulation  was  a  quick-turn  product  developed  by  integrating  existing 
applications  to  provide  a  baseline  from  which  predictive  capabilities  could  be  grown  and 
from  which  visualization  concepts  could  be  demonstrated.  The  spiral  one  BNet 
simulation  was  driven  by  a  static  Bayesian  Network  engine  and  thus  had  no  predictive 
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capability.  Future  spirals  were  designed  to  include  temporal  belief  networks  and  thus 
include  an  element  of  prediction. 

Fourth,  a  framework  was  developed  for  improving  RFIs  and  capturing  the  knowledge 
therein.  The  framework  was  envisioned  as  a  two-step  process  -  one  to  support  short-term 
RFIs  and  one  to  support  long-term  RFIs.  Short-term  RFIs  benefit  from  a  standardized 
RFI  process  to  capture  essential  information  elements.  The  inherent  specificity  provides  a 
more  precise  and  accurate  request  which  subsequently  enables  a  better,  targeted  response. 
Long-term  RFIs  are  more  analysis-focused  and  require  slightly  different  information.  A 
key  feature  for  both  RFI  types  is  the  ability  to  formulate  context,  i.e.  provide  the 
responder  a  basis  for  understanding  why  the  request  was  written,  what  research  had  been 
conducted,  and  what  product  attributes  the  requester  is  most  interested  in,  such  as  format, 
size,  timeliness,  and  classification. 

The  evolution  of  user  requirements  and  subsequent  visualizations  can  be  seen  in 
‘CDA4PBA  Visualizations.ppt.’ 
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6  Summary 

The  CDA4PBA  project  produced  findings  across  a  breadth  of  topics  areas  (with  respect 
to  PBA  in  a  military  operational  environment,  specifically  an  AOC).  A  number  of 
activities  could  benefit  from  continued  activity.  The  cognitive  analysis  was  extensive 
with  respect  to  the  organizations  interviewed,  specifically  the  AIS  and  SIS.  However,  a 
more  detailed  understanding  of  PBA  could  be  achieved  if  a  larger  cross-section  of 
organization  were  considered,  such  as  the  Information  Warfare  Flight  (IWF),  Strategy,  or 
Combat  Plans.  The  additional  analysis  could  also  be  used  to  support,  modify,  or  create 
additional  operational  requirements. 

The  groundwork  for  a  predictive  capability  has  been  formulated.  Much  work  remains  to 
fully  prove  these  capabilities,  however,  initial  incremental  steps  could  determine  whether 
the  Forecast  Model  is  viable  or  whether  a  temporal  belief  network  model  provides  a 
usable  predictive  capability,  particularly  with  respect  to  a  user’s  ability  to  populate  and 
operate  the  model.  Finally,  the  need  for  an  RFI  support  system  was  identified  throughout 
all  aspects  of  CDA4PBA.  The  ability  to  clearly  and  concisely  state  a  problem,  and  in  a 
manner  which  creates  an  environment  of  understanding  for  the  responder,  is  a  universal 
requirement. 
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7  Acronym  List 

AC2 

Aerospace  Command  &  Control 

ACWA 

Applied  Cognitive  Work  Analysis  (define  E*) 

AFB 

Air  Force  Base 

AOC 

Air  &  Space  Operations  Center 

BNet 

Bayesian  Belief  Network 

CDA4PBA 

Commander’s  Decision  Aid  for  Predictive  Battlespace  Awareness 

CIGAR 

Context-driven  InfoSpace  Configuration  for  Augmented  Cognitive 
Readiness 

COA 

Course  of  Action 

COTS 

Commercial  Off-The-Shelf 

CPS 

Combat  Plans  Squadron 

CTA 

Cognitive  Task  Analysis 

CWA 

Cognitive  Work  Analysis 

CWR 

Cognitive  Work  Requirement 

DARPA 

Defense  Advanced  Research  Projects  Agency 

EEI 

Essential  Element  of  Information 

EUCOM 

European  Command  (define  E') 

FAN 

Functional  Abstraction  Network 

FM 

Forecast  Model 

GOTS 

Government  Off-The-Shelf 

GPN 

Goal-Process  Node 

IRR 

Information  Relationship  Requirement 

ISR 

Intelligence,  Surveillance,  and  Reconnaissance 

IWE 

Information  Warfare  Effectiveness 

JFACC 

Joint  Forces  Air  Component  Commander 

KE 

Knowledge  Elicitation 

PBA 

Predictive  Battlespace  Awareness 

RFI 

Request  for  Information 

SmartRFI 

Smart  Request  for  Information 

SME 

Subject-Matter  Expert 

SRA 

SRA  International,  Inc. 

UML 

Unified  Modeling  Language 

USAF 

United  States  Air  Force 

WDAR 

Work-Domain  Analysis  Report  (define  E‘  occurrence) 
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Appendix  I:  CDA4PBA  Knowledge  Elicitation  Methodology 

Introduction 

The  objective  of  our  Knowledge  Elicitation  (KE)  is  to  gather,  through  a  combination  of 
methods,  a  complementary  set  of  information  about  the  decision-making  problem  space 
under  consideration.  Typically,  the  work  domain  analysis  is  performed  based  on  a 
variety  of  KE  activities.  This  involves  interactions  with  expert  practitioners  in  the 
domain  and  includes  face-to-face  interviews  with  the  experts,  watching  the  experts  work 
in  the  domain,  verbal  protocol  techniques,  and  other  Cognitive  Task  Analysis  (CTA)  and 
Cognitive  Work  Analysis  (CWA)  methods  (cf  Potter,  Roth,  Woods  &  Elm,  2000; 
Vicente,  1999).  In  practice,  this  is  an  iterative,  progressively  deepening  process. 

The  phrase  ‘bootstrapping  process’  has  been  used  to  describe  this  process  and  emphasize 
the  fact  that  the  process  builds  on  itself  (Potter,  et  ah,  2000).  Each  step  taken  expands  the 
base  of  knowledge  providing  opportunity  to  take  the  next  step.  Making  progress  on  one 
line  of  inquiry  (understanding  one  aspect  of  the  field  of  practice)  creates  the  room  to 
make  progress  on  another.  One  starts  from  an  initial  base  of  knowledge  regarding  the 
domain  and  how  practitioners  function  within  it  (often  very  limited).  One  then  uses  a 
number  of  KE  techniques  to  expand  on  and  enrich  the  base  understanding  and  evolve  an 
ACWA  model  from  which  ideas  for  improved  support  can  be  generated.  For  example, 
one  might  start  by  reading  available  documents  that  provide  background  on  the  field  of 
practice  (e.g.,  training  manuals,  procedures),  the  knowledge  gained  will  raise  new 
questions  or  hypotheses  to  pursue  that  can  then  be  addressed  in  interviews  with  domain 
experts,  it  will  also  provide  the  background  for  interpreting  what  the  experts  say.  In  turn, 
the  results  of  interviews  or  exercises  may  point  to  complicating  factors  in  the  domain  that 
need  to  be  modeled  in  more  detail  in  the  FAN.  This  provides  the  necessary  background 
to  create  scenarios  to  be  used  to  observe  practitioner  performance  under  simulated 
conditions  or  to  look  for  confirming  example  cases  or  interpret  observations  in 
naturalistic  field  studies. 

The  selection  of  which  technique(s)  to  use  and  how  many  techniques  to  employ  is 
motivated  by  the  need  to  produce  a  model  of  the  field  of  practice  and  how  domain 
practitioners  operate  in  that  field.  In  practice  the  modeling  process  generally  requires  the 
use  of  multiple  converging  techniques  that  include  techniques  that  focus  on 
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understanding  the  domain  demands  as  well  as  techniques  that  focus  on  understanding  the 
knowledge  and  strategies  of  domain  practitioners.  The  KE  process  is  highly 
opportunistic.  The  particular  set  of  techniques  selected  will  be  strongly  determined  by 
the  pragmatics  of  the  specific  local  conditions.  Typically,  access  to  domain  practitioners 
is  limited.  In  that  case  other  sources  of  domain  knowledge  (e.g.  written  documents)  can 
be  maximally  exploited  before  turning  to  domain  experts.  In  some  cases  observing 
domain  experts  in  actual  work  practice  (e.g.,  using  ethnographic  methods  or  simulator 
studies)  may  be  impractical,  in  those  cases  using  structured  interview  techniques  may  be 
the  most  practical  methods  available.  In  cases  where  domain  experts  are  not  accessible  at 
all  (e.g.,  in  highly  classified  government  applications),  it  becomes  necessary  to  look  for 
surrogate  experts  (e.g.,  individuals  who  have  performed  the  task  in  the  past)  or  analogous 
domains  to  examine. 

One  key  element  of  the  Knowledge  Elicitation  is  that  it  is  performed  iteratively  with  the 
ACWA  effort.  As  interim  results  from  the  modeling  task  become  available,  they  will  be 
used  as  material  for  further  elicitation.  The  process  of  constructing  the  ACWA  artifacts 
provides  requirements  for  the  information  needed  to  enrich  the  model.  As  an  additional 
benefit,  the  Functional  Abstraction  Network  model  has  been  shown  to  be  extremely 
powerful  in  integrating  seemingly  disparate  sources  of  information  from  a  KE  process 
into  a  unified  analysis  and  design  framework.  Thus,  the  focus  of  this  KE  task  is  to,  in  an 
iterative  and  participatory  manner,  provide  the  data  necessary  to  construct  the  set  of 
ACWA  artifacts. 

As  mentioned  above,  our  KE  approach  will  utilize  a  combination  of  methods  to  achieve 
the  desired  result.  These  will  need  to  include: 

•  Reviewing  relevant  technical  documentation  on  PBA; 

•  Discussions  with  SMEs  on  difficult  PBA  scenarios  and  how  they  impact  the 

decision-making  process; 

•  Discussions  with  SMEs  on  our  FAN  (and  associated  cognitive  and  information 
requirements)  and  its  gaps  in  modeling  the  work  domain;  and 

•  Observations  of  simulation  and  training  exercises  related  to  PBA  in  order  to 
operationally  validate  (or  negate)  the  analytical  findings. 

These  different  activities  are  laid  out  in  the  following  plan. 
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KE  Data  Collection  Plan 

Each  of  the  steps  in  our  ACWA  methodology  is  'fed'  by  a  variety  of  Knowledge 
Elicitation  activities.  This  will  involve  interactions  with  expert  practitioners  in  the 
domain  in  a  variety  of  forms  (e.g.,  face-to-face  interviews,  unstructured  observations  in 
actual  work  situations,  and  structured  observations  in  simulated  conditions).  We  will 
utilize  a  combination  of  staff  and  consultant  subject  matter  experts  to  complement  'in  the 
field'  KE  data  collection  activities. 

The  focus  of  the  KE  effort  will  be  to: 

•  Gain  familiarization  and  understanding  of  the  goals  and  cognitive  work 
requirements; 

•  Ground  this  understanding  in  terms  of  the  decision-maker's  work  context;  and 

•  Gain  a  solid  understanding  of  the  actual  work  that  is  required  to  conduct  and 
maintain  the  PBA  process. 

Initial  Familiarization 

This  phase  consists  of  reviews  of  related  documentation  (including  training  material, 
system  descriptions,  and  operational  scenarios),  initial  interviews  with  SMEs,  and  task 
walkthroughs.  These  activities  provide  the  background  knowledge  to  understand  the 
tasks  to  be  performed  by  the  operators  and  their  comments,  but  typically  do  not  provide 
the  critical  insights  into  what  makes  the  decision-making  especially  difficult. 

The  accompanying  list  of  documentation  itemizes  the  material  we  have  available  for 
literature  review.  We  will  need  SMEs  to  review  this  list  and  provide  additional 
documentation  for  use  in  the  initial  familiarization  phase. 

In  addition,  we  will  need  a  one  or  two  day  meeting  with  an  SME  from  the  work  team  to 
provide  initial  insights  to  guide  our  activities. 

Initial  Functional  Modeling  of  the  Domain 

Based  on  the  above,  an  initial  functional  model  of  the  problem  space  can  be  developed. 
This  provides  a  starting  point  for  understanding  fundamental  relationships,  scope  of  the 
problem  space  to  be  modeled,  and  essential  objectives.  Even  an  initial  austere 
representation  can  convey  the  structure  of  entities  that  relate  to  more  abstract  concepts. 
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Most  importantly,  this  representation  provides  a  framework  for  preparing  for  future 
interviews  (based  on  gaps  in  the  initial  functional  model)  as  well  as  interpreting  results  of 
those  interviews  and  observations  and  integrating  those  results  into  a  unified,  well- 
grounded  representation. 

We  will  need  occasional  meetings  (mostly  telecon  with  possible  face-to-face)  with  an 
SME  from  the  work  team  to  provide  initial  feedback  and  insights  to  our  analysis 
activities. 

Interviews  with  Subject  Matter  Experts 

Based  on  the  initial  model,  areas  for  further  investigation  can  be  identified  which  frame 
the  lines  of  questioning  to  be  pursued.  These  interviews  then  focus  on  understanding  the 
factors  in  the  domain  that  makes  the  particular  tasks  difficult  and  the  knowledge  and  skill 
requirements  for  expert  performance.  These  interviews  can  take  a  variety  of  forms: 

•  One-on-one  interviews  with  SMEs  where  the  format  follows  a  question-and- 
answer  approach;  or 

•  Group  interviews  or  'expert  panel'  sessions  where  the  format  focuses  on 
facilitating  discussions  between  the  multiple  SMEs  to  highlight  different 
perspectives  to  the  problem. 

Therefore,  we  are  flexible  with  respect  to  the  specific  availability  of  the  SMEs.  In 
general,  'in  the  field'  KE  events  are  scheduled  for  2-4  days,  depending  on  the  specific 
availability  of  the  SMEs.  If  the  set  is  available  as  a  single  group,  the  event  should  be 
only  2  days.  However,  if  the  set  is  available  individually,  the  event  should  be  longer  in 
order  to  accommodate  this  schedule.  The  specific  scheduling  of  these  interviews  will 
depend  primarily  on  the  availability  of  the  SMEs. 

With  respect  to  the  SME  needs,  in  general  the  SME  requirements  include: 

•  Operationally  current  practitioners  who  have  a  deep  understanding  of  the 
processes  and  the  underlying  basis  for  these  processes; 

•  Tactics  /  doctrine  experts; 

•  Instructors  within  the  specific  problem  space. 

We  will  need  an  SME  from  the  work  team  to  attend  these  interviews  and  provide 
'translation  services'  for  unfamiliar  domain  specific  concepts. 
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Interim  Functional  Modeling 

The  results  of  the  above  tasks  will  be  used  as  refinements  and  expansions  to  the  ACWA 
artifaets  that  were  developed  earlier.  This  analysis  effort  will  provide  the  deeision- 
centered  framework  for  additional  KE  aetivities.  Specifically,  the  focus  often  shifts  to 
the  design  of  scenarios  to  be  potentially  included  in  the  observation-focused  KE 
activities. 

We  will  need  occasional  meetings  (mostly  telecon  with  possible  face-to-face)  with  an 
SME  from  the  work  team  to  provide  initial  feedback  and  insights  to  our  analysis 
activities. 

Observation  of  Performance  in  Simulated  Conditions 

With  the  baseline  knowledge  acquired  in  the  first  three  phases,  a  valuable  complementary 
KE  activity  is  the  observation  of  behavior  under  realistic  dynamic  conditions.  This  can 
provide  converging  evidence  of  the  validity  and  effectiveness  of  the  strategies  they 
described  during  the  interview  sessions.  This  can  also  provide  an  opportunity  to  reveal 
additional  expert  strategies  that  were  triggered  by  the  context  of  actual  task  performance 
that  might  otherwise  have  remained  inert.  Ideally  this  involves  some  level  of  control 
over  the  scenarios  presented  to  the  operators. 

We  will  need  an  SME  from  the  work  team  to  attend  these  interviews  and  provide 
'translation  services'  for  unfamiliar  domain  specific  concepts. 
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Appendix  II:  CDA4PBA  Forecast  Model  Use  Case  Descriptions 

The  forecast  model  (FM)  is  a  process  for  populating  and  updating  one  or  more  selected 
military  models  and  instantiating  or  constructing  those  models  with  intelligence 
information.  The  degree  to  which  intelligence  information  fits  the  models  determines 
how  “good”  the  FM  prediction  is.  (Note:  “good”  from  the  viewpoint  of  being  supported 
by  data.  An  assumption  is  made  that  the  input  models  are  valid  and  accurate.) 

FM  processing  results  in  1)  RFIs  issued  based  on  geo-spatial,  temporal  and  semantic 
quality  information  and  2)  useful  information  containing  which  otherwise  may  not  have 
been  processed  due  to  a  lack  of  context. 

The  basic  sequence  of  activities  starts  with  a  commander  requesting  an  update  to  the 
“forecast”  for  a  specific  event.  The  event  has  already  been  identified,  for  example  a 
trigger  event  is  formulated  in  the  plan.  Information  sources  such  as  USMTF  or  WSV  are 
selected  to  support  the  event.  A  semantic  search  (noun/verb  phases  in  context)  is 
conducted  on  the  information  sources  and  results  are  stored  in  a  database.  Noun/verb 
phrases  are  correlated  in  space  (geo-location)  and  time  (temporal).  Models  are  created  or 
updated  and  information  source  processing  continued  to  iteratively  develop  the  models. 
Match  attributes.  A  graphical  model  is  developed.  The  model(s)  are  evaluated  for  causal 
relationships  (correlated  links  are  converted  to  causal  where  evidence  suggests  it  is 
appropriate).  The  model(s)  are  further  evaluated  for  dependency  relationships  (correlated 
links  are  converted  to  dependency  where  evidence  suggests  it  is  appropriate).  Throughout 
this  process,  RFIs  are  issued  and  analyzed  to  further  refine  the  model(s).  A  Goodness  of 
Fit  (GOF)  test  is  performed  at  any  point  in  the  process  and  on  any  subset  of  links/nodes 
therein. 

The  high-level  use  case  descriptions  are  provided  below.  The  details  associated  with  each 
use  case  are  contained  in  the  file  ‘SRA  FM  User  Interactions  I3Jun05  v3.doc’. 
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Use  Case  1  —  Analyst  selects  intelligence  information  inputs  for  FM  processing 

Context:  The  analyst  must  identify  information  sources  relevant  to  the  Commander’s 
request. 

Goal:  The  analyst  selects  information  sources  (Intel  or  other  information)  to  either 

instantiate  or  construct  the  event  model(s). 

Success:  Information  sources  have  been  selected  for  FM  processing. 

Use  Case  2- Analyst  selects  model  information  for  FM  processing 

Context:  The  analyst  must  identify  Models  relevant  to  the  Commander’s  request. 

Goal:  The  analyst  selects  Model(s)  appropriate  for  the  trigger  event. 

Success:  Models  have  been  selected  for  FM  processing. 

Use  Case  3  —  Analyst  selects  model  attribute  information  for  FM  processing 

Context:  The  analyst  must  identify  Model  Attributes  relevant  to  the  Commander’s 
request. 

Goal:  The  analyst  selects  Model  Attributes  appropriate  for  the  trigger  event. 

Success:  Model  Attributes  have  been  selected  for  FM  processing. 

Use  Case  4  —  FM  processing  step  to  associate  trigger  event  Attributes  with  Intel 

Context:  FM  1)  matches  Model  attributes  to  data  in  the  selected  Intel  reports  and  2) 
correlates  “other  data  elements’’  in  time  and  space. 

Goal:  Match  Model  Attributes  to  those  found  in  Intel  and  identify  data  correlations 

in  time  and  space. 

Success:  Depends  upon  Goodness  of  Fit  results. 

Use  Case  5  -  Goodness  of  Fit  test  between  trigger  event  Attributes  and  Intel  data 

Context:  The  Goodness  of  Fit  (GOF)  can  be  conducted  at  any  point  in  the  process,  on 
the  Model  or  Attributes,  and  on  any  subset  thereof  This  step  quantifies  how 
well  Model  Attributes  match  data  in  the  selected  Intel  reports.  Isolating  the 
Model  for  a  GOF  can  help  in  issuing  and  ranking  RFIs.  Smart  RFIs  are  issued 
to  1)  improve  link  quality,  2)  identify  “unknown’’  information,  3)  add  to  the 
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Model  or  Attributes,  and  4)  add  to  or  improve  pre-processing  results  (quantity 
or  quality). 

Goal:  Match  Model  Attributes  to  those  found  in  Intel  reports. 

Success:  Goodness  of  Fit  meets  or  exceeds  user-specified  criteria.  The  “best”  Model  is 
selected. 

Use  Case  6  -  Build  graphical  model  of  causal,  dependency  and  correlation 

relationships 

Context:  Build  the  GM  and  populate  with  data  found  in  the  Intel  reports.  Analyze  the 
Model  for  True  (genuine),  Potential  (probably).  Spurious  associations,  and 
Unknown  Causal  relationships  (links)  as  well  as  Dependency  relationships  for 
unobserved  (latent)  and  observed  nodes.  Causal  relationships  provide:  1)  a 
sequence  in  time,  i.e.,  A  must  happen  before  B,  if  A  causes  B,  2)  the  state  of 
one  node  causes  a  change  in  state  of  another  node,  and  3)  causal  implies  a 
dependency  relationship  (no  loops  though). 

Goal:  Interpret  the  GM  with  respect  to  Causal  and  Dependency  relationships 

Success:  User  understands  extent  and  magnitude  of  Causal  and  Dependency 
relationships. 

Use  Case  7  -  Discrete  Event  Simulator  (DES)  to  forecast  at  a  specific  point  in  time 

Context:  Run  the  GM  through  a  DES  to  determine  when  forecast  will  occur  and 
information  requirements  which  may  be  necessary  to  support  the  forecast. 

Goal:  Understand  within  model-determined  confidence  limits  when  the  forecast  will 

occur. 

Success:  User  provides  a  forecast  with  an  overall  confidence  level. 
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Appendix  III:  CDA4PBA  Operational  Requirements  per  the  WDAR 

GPN  1.1  Successfully  Manage  Air  &  Space  Objectives 


Req  ID 

Requirement 

Agent 

1.1-01 

The  system  shall  display  the  focus  of  individual  Air  & 
Space  Objectives  over  time 

Support  Staff 

1.1-02 

The  system  shall  display  the  focus  of  the  entire  set  of  Air 
&  Space  Objectives  over  time 

Support  Staff 

1.1-03 

The  system  shall  display  the  relationships  between  Air  & 
Space  Objectives 

Support  Staff 

1.1-04 

The  system  shall  display  the  expected  Air  &  Space 
Objective  satisfaction/end  time 

Commander  Staff 

1.1-05 

The  system  shall  display  the  expected  Air  &  Space 
Objective  satisfaction/end  time  in  parallel  with  the  actual 
satisfaction  state 

Commander  Staff 

1.1-06 

The  system  shall  display  the  amount  of  risk  in  completing 
individual  Air  &  Space  Objectives  based  on  the  current 
situation 

Support  Staff 

1.1-07 

The  system  shall  display  the  confidence  in  the  ability  to 
complete  the  individual  Air  &  Space  Objectives  by  the 
estimated  end  time. 

Support  Staff 

1.1-08 

The  system  shall  display  any  changes  in  guidance  and 
intent  over  time 

Support  Staff 

1.1-09 

The  system  shall  display  the  achievement/completion 
status  for  individual  objectives  over  time 

Commander  Staff 

1.1-10 

The  system  shall  display  the  actual 
achievement/completion  status  for  individual  objectives  in 
parallel  to  projected  achievement/completion  status  over 
time 

Commander  Staff 

1.1-11 

The  system  shall  display  the  confidence  in  the 
assessments  of  the  individual  Objective's  statuses. 

Commander  Staff 

1.1-12 

The  system  shall  display  the  valuated  indicators  that  have 
been  bundled  for  each  Air  &  Space  Objective 

Commander  Staff 

1.1-13 

The  system  shall  display  the  reliability  and  the  timeliness 
(its  utility)  of  each  indicator  within  a  specific  Air  &  Space 
Objective's  bundle. 

Support  Staff 

GPN  2.1 

Successfully  Manage  Air  &  Space  Effects 

Req  ID 

Requirement 

Agent 

2.1-01 

The  system  shall  display  the  relationships  and 

Commander 

dependencies  (sequential,  temporal  and  simultaneous) 
between  the  entire  Air  &  Space  Effects  network. 

Staff 

2.1-02 

The  system  shall  display  the  achievement/success 

Commander 

status  of  each  Air  &  Space  Effect  within  the  context  of 
all  other  Effects'  status  over  time. 

Staff 

2.1-03 

The  system  shall  display  the  priority  of  each  Air  & 

Commander 

Space  Effect  within  the  entire  set 

Staff 
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2.1-04 

The  system  shall  display  the  weight  of  each  of  the  Air 
&  Space  Effects'  status  as  they  support  the  Air  & 

Space  Objectives  status 

Commander 

Staff 

2.1-05 

The  system  shall  house  an  algorithm  that  provides  the 
"overall  score"  of  the  Air  &  Space  Effect  Network 

Support  Staff 

2.1-06 

The  system  shall  display  the  "overall  score"  in  parallel 
with  the  set  of  individual  Effects'  status  that  make  up 
the  "overall  score". 

Commander 

Staff 

2.1-07 

The  system  shall  display  the  linkage  between  the  Air  & 
Space  Effects  and  specific  posited  adversary 
capabilities 

Commander 

Staff 

2.1-08 

The  system  shall  display  the  linkage  between  the  Air  & 
Space  Effects  and  specific  inferred  adversary  will 

Commander 

Staff 

2.1-09 

The  system  shall  display  the  reliability  and  the 
timeliness  (its  utility)  of  each  indicator  within  a  specific 

Air  &  Space  Effect's  bundle. 

Support  Staff 

2.1-10 

The  system  shall  display  the  resource  cost  for  the 
potential  collection  of  each  indicator  within  a  specific 

Air  &  Space  Effect's  bundle. 

Support  Staff 

2.1-11 

The  system  shall  allow  the  user  to  prioritize  the 
indicators  within  a  specific  Air  &  Space  Effect  based  on 
their  utility  and  by  the  number  of  times  a  single 
indicator  is  used  for  multiple  Effects. 

Support  Staff 

2.1-12 

The  system  shall  display  the  number  of  times  an 
indicator  is  used  across  the  entire  Air  &  Space  Effect 
network. 

Support  Staff 

2.1-13 

The  system  shall  display  the  achievement/success 
status  of  each  Air  &  Space  Effect  as  it  is  supported  by 
the  execution/success  status  (or  lack  thereof)  of 
related  Air  &  Space  Aims. 

Commander 

Staff 

2.1-14 

The  system  shall  display  adversary  capability  level 
prior  to  Air  &  Space  Aim  execution 

Commander 

Staff 

2.1-15 

The  system  shall  display  the  degree  of  projected 
degredation  to  the  adversary  capability  after  the  Air  & 
Space  Aim  execution 

Commander 

Staff 

2.1-16 

GPN  3.1 

The  system  shall  display  the  rate  in  which  the 
adversary  can  reconstitute  a  specific  adversary 
capability 

Successfully  Manage  Air  &  Space  Aims 

Support  Staff 

Req  ID 

Requirement 

Agent 

3.1-01 

The  system  shall  display  the  execution  status  of  the 
entire  set  of  Aims  over  time. 

Commander 

Staff 

3.1-02 

The  system  shall  display  the  planned  execution  time  for 
each  individual  Aim  in  coordination  with  the  actual 
execution  time  for  each  individual  Aim. 

Commander 

Staff 

3.1-03 

The  system  shall  display  the  success  status  of  the 
entire  set  of  Aims  over  time. 

Commander 

Staff 
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3.1-04 

The  system  shall  display  the  actual  execution  status  for 
each  individual  Aim. 

Commander 

Staff 

3.1-05 

The  system  shall  display  the  relationship  between  the 

Air  &  Space  effects  and  each  individual  aim 

Support  Staff 

3.1-06 

The  system  shall  display  the  execution  status  of  the 

Aims  related  to  a  specific  Air  &  Space  effect's  success 
status. 

Commander 

Staff 

3.1-07 

The  system  shall  display  the  success  status  of  the 

Aims  related  to  a  specific  Air  &  Space  effect's  success 
status. 

Commander 

Staff 

3.1-08 

GPN  3.2 

The  system  shall  display  the  target's  affordances,  the 
types  of  combat  power  that  would  be  effective  against  it 
and  the  availability  of  all  types  of  Air  &  Space  Combat 
Power 

Successfully  Infer  Adversary  Goals 

Support  Staff 

Req  ID 

Requirement 

Agent 

3.2-01 

The  system  shall  display  the  success  status  of 
adversary  goals  over  time 

Commander 

Staff 

3.2-02 

The  system  shall  display  the  current  indicator  set  and 
their  status  as  related  to  the  success  status  of  each 
adversary  goal 

Support  Staff 

3.2-03 

The  system  shall  display  the  completion  status  of 
adversary  goals  over  time 

Commander 

Staff 

3.2-04 

The  system  shall  display  the  estimated  progress  of  the 
adversary  toward  their  goals  along  with  the  end  state  of 
their  inferred  goals 

Commander 

Staff 

3.2-05 

The  system  shall  display  the  delta  between  expected 
progress  and  actual  progress  towards  an  inferred 
adversary  goal 

Commander 

Staff 

3.2-06 

The  system  shall  display  the  temporal  delta  between 
expected  progress  and  actual  progress  towards  an 
inferred  adversary  goal 

Commander 

Staff 

3.2-07 

The  system  shall  show  the  relationships  between  the 
inferred  adversary  goals  and  the  assumed  adversary 
will  (by  societal  group  that  applies) 

Commander 

Staff 

3.2-08 

The  system  shall  show  the  relationships  between  the 
inferred  adversary  goals  and  the  inferred  adversary 
capabilities  (by  capability  types  that  apply) 

Commander 

Staff 

3.2-09 

The  system  shall  show  changes  to  assumed  adversary 
will  as  they  relate  to  the  inferred  adversary  goals. 

Commander 

Staff 

3.2-10 

The  system  shall  arrange  the  inferred  adversary  goals 
in  order  of  their  potential  ability  to  prevent  Air  &  Space 
objectives 

Commander 

Staff 

3.2-11 

The  system  shall  arrange  the  inferred  adversary  goals 
in  order  of  their  potential  ability  to  enable  Air  &  Space 
objectives 

Commander 

Staff 
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GPN  3.3 

Req  ID 

3.3- 01 

3.3- 02 

3.3- 03 

3.3- 04 

3.3- 05 

3.3- 06 

3.3- 07 

3.3- 08 

3.3- 09 

3.3- 10 

3.3- 11 

3.3- 12 

3.3- 13 

3.3- 14 


Successfully  Assess  Adversary  Behaviors 
Requirement 

The  system  shall  illustrate  the  actual  adversary 
behavior  in  contrast  to  the  expected  adversary 
behavior  overtime. 

The  system  shall  display  the  difference  between  the 
time  of  actual  adversary  behavior  in  line  with  the  time 
of  expected  adversary  behavior. 

The  system  shall  provide  the  user  with  the  ability  to 
assign  metrics  for  determining  the  supporting  nature 
between  adversary  behavior  and  adversary 
capabilities. 

The  system  shall  provide  the  user  with  the  ability  to 
assign  metrics  for  determining  the  supporting  nature 
between  adversary  behavior  and  adversary  will. 

The  system  shall  display  the  correlation  between 
adversary  behavior  and  adversary  capabilities  over 
time. 

The  system  shall  display  the  correlation  between 
adversary  behavior  and  adversary  will  over  time. 

The  system  shall  display  the  relationships  between  the 
adversary  behavior  and  the  inferred  adversary  goals 
while  displaying  the  adversary  goals'  success  status. 

The  system  shall  display  the  assessed  adversary 
behavior  events  along  side  of  the  execution  status  of 
Air  and  Space  Aims. 

The  system  shall  display  the  estimated  relationship 
strength  between  an  adversary  behavior  and  Air  & 
Space  Aims. 

The  system  shall  track  the  accuracy  of  the  expected 
adversary  behavior  compared  to  the  actual  adversary 
behavior 

The  system  shall  provide  a  means  for  the  user  to 
illustrate  the  expected  adversary  behaviors  as  they  are 
derived  from  inferred  adversary  goals  by  their 
hierarchical/organizational  relationships 

The  system  shall  provide  a  means  for  the  user  to 
illustrate  the  expected  adversary  behaviors  as  they  are 
derived  from  inferred  adversary  goals  by  the  position  of 
adversary  forces  in  parallel  with  Air  &  Space  Forces. 


The  system  shall  provide  a  means  for  the  user  to 
illustrate  the  expected  adversary  behaviors  as  they  are 
derived  from  inferred  adversary  goals  by  their 
sequential/temporal  relationships 

The  system  shall  display  the  observed  adversary 
behavior  differently  than  the  expected  adversary 
behavior. 


Agent 

Commander 

Staff 

Commander 

Staff 


Commander 

Staff 


Commander 

Staff 


Commander 

Staff 


Commander 

Staff 

Commander 

Staff 


Support  Staff 


Support  Staff 


Commander 

Staff 


Support  Staff 


Support  Staff 


Support  Staff 


Support  Staff 
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3.3- 15 

GPN  3.4 

Req  ID 

3.4- 01 

3.4- 02 

3.4- 03 

3.4- 04 

3.4- 05 

3.4- 06 

3.4- 07 

3.4- 08 

3.4- 09 

3.4- 10 

3.4- 11 

3.4- 12 

3.4- 13 

3.4- 14 

GPN  3.6 

Req  ID 

3.6-01 


The  system  shall  allow  the  user  to  link  intelligence 
results  regarding  motivation  for  expected  and  observed 
adversary  behavior. 

Successfully  Posit  Adversary  Capabilities 
Requirement 

The  system  shall  display  the  entire  posited  capability 
network. 

The  system  shall  distinctly  display  the  various  types  of 
inferred  adversary  capabilities  (Political,  Military, 
Economic,  Social,  Informational,  Infrastructure, 
Humanitarian,  etc) 

The  system  shall  display  the  linkage  between  assessed 
adversary  behaviors^ctions  which  demonstrate  certain 
capabilities 

The  system  shall  allow  the  user  to  enter  and  track  a 
confidence  level  for  Inferences  made  about  each  type 
of  adversary  capability  over  time. 

The  system  shall  allow  the  user  to  tie  inferred 
adversary  capabilities  with  supporting  Intel  results 
The  system  shall  show  the  degree  to  which  the  posited 
adversary  capabilities  support  the  achievement  of 
adversary  goals. 

The  system  shall  show  the  relationship  between  the 
posited  adversary  capabilities  and  the  Air  &  Space 
Effect's  network 

The  system  shall  provide  the  relevance  of  assessed 
adversary  behaviors  as  related  to  inferred  capabilities. 
The  system  shall  provide  the  quality  of  evidence  that 
supports  the  relevance  rating  given  to  the  relationships 
between  adversary  behaviors  and  inferred  capabilities. 

The  system  shall  alert  the  user  when  new  intelligence 
results,  specific  to  adversary  capabilities  or  capability 
type,  are  ready  to  be  reviewed. 

The  system  shall  track  the  use  of  specific  capabilities 
over  time. 

The  system  shall  track  changes  in  effectiveness  of 
specific  capabilities  over  time. 

The  system  shall  track  changes  in  efficiency  of  system 
of  specific  capabilities  over  time. 

The  system  shall  show  the  age  of  specific  adversary 
capabilities  over  time. 

Successfully  Infer  Adversary  Will 

Requirement 

The  system  shall  display  the  number  and  identity  of  Air 
&  Space  Effects  that  were  created  based  on  inferences 
regarding  adversary  will. 


Support  Staff 

Agent 

Support  Staff 
Support  Staff 

Support  Staff 
Support  Staff 

Support  Staff 

Commander 

Staff 

Commander 

Staff 

Support  Staff 
Support  Staff 

Support  Staff 

Support  Staff 
Support  Staff 
Support  Staff 
Support  Staff 

Agent 

Commander 

Staff 
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3.6-02 

The  system  shall  track  the  actor(s)  (individual,  group, 
organization,  etc)  whose  will  is  being  monitored  as 
related  to  specific  Air  &  Space  Effects. 

Commander 

Staff 

3.6-03 

The  system  shall  monitor  the  coherence  of  will  across 
the  specific  actor(s)  (individual,  group,  organization, 
etc)  as  related  to  specific  Air  &  Space  Effects. 

Commander 

Staff 

3.6-04 

The  system  shall  display  the  degree  to  which  an 
inferred  adversary  will  could  (hypothesized)  prevent  an 

Air  &  Space  Effect. 

Commander 

Staff 

3.6-05 

The  system  shall  display  the  degree  to  which  an 
inferred  adversary  will  could  (hypothesized)  enable  an 

Air  &  Space  Effect. 

Commander 

Staff 

3.6-06 

The  system  shall  display  changes  in  the  strength  of  the 
adversary's  will  in  parallel  with  the  execution  and 
success  status  of  Air  &  Space  Effects  over  time. 

Commander 

Staff 

3.6-07 

The  system  shall  provide  the  means  to  tie  intelligence 
results  to  any  point  in  time  when  the  changes  in 
strength  occur. 

Support  Staff 

3.6-08 

The  system  shall  provide  the  means  to  search  through 
historical  sources  based  on  the  attributes  of  the  current 
conditions 

Support  Staff 

3.6-09 

The  system  shall  display  the  relationships  between  the 
will  of  multiple  segments  of  society  (Political  leadership. 
Military  leadership.  Front  Line  soldiers,  civilian 
population  soldiers,  etc.)  overtime. 

Support  Staff 

3.6-10 

The  system  shall  provide  the  status  of  the  will  of  the 
Political  Leadership  over  time. 

Commander 

Staff 

3.6-11 

The  system  shall  provide  the  status  of  the  will  of  the 
Military  Leadership  over  time. 

Commander 

Staff 

3.6-12 

The  system  shall  provide  the  status  of  the  will  of  the 

Front  Line  soldiers  over  time. 

Commander 

Staff 

3.6-13 

The  system  shall  provide  the  status  of  the  will  of  the 
civilian  population  soldiers  over  time. 

Commander 

Staff 

3.6-14 

The  system  shall  display  the  level  of  risk  the  adversary 
is  willing  to  assume,  organized  by  societal  segment. 

Commander 

Staff 

3.6-15 

GPN  4.4 

The  system  shall  provide  the  means  to  link  specific 
intelligence  results  with  assumptions  made  about  the 
adversary  will. 

Successfully  Manage  Indicators 

Support  Staff 

Req  ID 

Requirement 

Agent 

4.4-01 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Air  &  Space 
Objectives. 

Intel 

4.4-02 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Air  &  Space  Effects. 

Intel 
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4.4-03 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Adversary  Goals. 

Intel 

4.4-04 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Adversary  will. 

Intel 

4.4-05 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Adversary 
Capabilities. 

Intel 

4.4-06 

The  system  shall  display  and  arrange  the  Indicators 
based  on  their  relevancy  to  current  Adversary 

Behavior. 

Intel 

4.4-07 

The  system  shall  display  the  1  to  many  number  of  uses 
an  indicator  may  have  across  a  plan 

Intel 

4.4-08 

The  system  shall  allow  the  user  to  view  the 
assessments  of  an  indicator's  quality  over  time 

Intel 

4.4-09 

The  system  shall  allow  the  user  to  establish  a 
maximum  time  duration  threshold  for  an  indicator's 
quality  since  it  was  last  assessed. 

Intel 

4.4-10 

The  system  shall  display  the  amount  of  time  since  the 
indicator's  quality  was  last  assessed. 

Intel 

4.4-11 

The  system  shall  provide  an  average  fulfillment  time 
estimate  for  the  collection  of  information  to  valuate  all 
indicators  within  a  bundle. 

Intel 

4.4-12 

The  system  shall  provide  a  resource  cost  estimate  for 
the  collection  of  information  to  valuate  all  indicators 
within  a  bundle. 

Intel 

4.4-13 

The  system  shall  display  the  coverage  a  new  indicator 
shall  provide  in  parallel  with  the  current  indicator's 
coverage  for  the  item  (Adv.  Goal,  Objective,  Effect,  etc) 
in  question. 

Intel 

4.4-14 

The  system  shall  display  the  priority  and  criticality  of 
the  selected  plan  element  in  question  (Objective, 

Effect,  etc). 

Intel 

4.4-15 

The  system  shall  provide  a  history  of  use  (frequency, 
level  of  plan,  last  date  of  use)  for  any  indicator  that  has 
been  used  previously. 

Intel 

GPN  6.3 

Successfully  Manage  Intelligence  Results 

Req  ID 

Requirement 

Agent 

The  system  shall  display  the  success  status  of  each 

Commander 

Intelligence  Requests  as  it  corresponds  to  the  priority 

Staff 

6.3-1 

driven  by  the  Commander's  intent 

The  system  shall  allow  the  user  to  select  the  temporal 

Commander 

duration  to  view  the  success  status  of  the  entire  set  of 

Staff 

6.3-2 

Intelligence  Requests  (hourly,  daily,  weekly,  etc) 

The  system  shall  allow  the  user  to  enter  new 

Intelligence  Requests  or  revise  existing  Intelligence 

Support  Staff 

6.3-3 

Requests  to  increase  the  clarity  of  the  request 

The  system  shall  show  the  collection  plan  in  contrast  to 

Support  Staff 

6.3-4 

the  intelligence  requests'  priority 
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The  system  shall  allow  the  user  to  assign  an 
uncertainty  (percentage)  of  an  Intelligence  result  as  a 

Support  Staff 

6.3-5 

function  of  the  clarity  of  the  Intelligence  Request 

The  system  shall  display  the  relationship  between  each 

Commander 

intelligence  result  as  it  relates  to  entire  set  of 

Staff 

6.3-6 

intelligence  requests. 

The  system  shall  allow  the  user  to  prioritize  "requests 

Support  Staff 

6.3-7 

for  analysis"  based  on  the  Commander's  requests. 

The  system  shall  group  the  Intelligence  Requests 
based  on  the  country  in  question's  Master,  Monitor  and 

Support  Staff 

6.3-8 

Measure  classification. 

The  system  shall  show  the  time  pressure  (pressure  to 
complete  intelligence  analysis)  on  each  individual 

Support  Staff 

6.3-9 

Intelligence  Requests. 

The  system  shall  direct  emphasis  on  Intelligence 
Requests  that  correlate  with  analysts'  specialized 

Intel 

6.3-10 

knowledge  bases. 

The  system  shall  display  the  initial  request  time  and  the 
expected  completion  time  for  each  Intelligence 

Intel 

6.3-11 

Request. 

The  system  shall  display  the  expected  completion  time 
in  contrast  to  the  actual  completion  time  for  each 

Support  Staff 

6.3-12 

Intelligence  Request. 

The  system  shall  provide  a  means  to  search  the 
intelligence  results  before  tasking  a  new  collection 

Support  Staff 

6.3-13 

effort. 

GPN  7.4 

Successfully  Provide  Applied  Collection  Power 
Findings 

Req  ID 

Requirement 

Agent 

7.4-01 

The  system  shall  allow  the  user  to  prioritize  their 
available  Intelligence  assets  based  on  their  collection 
effectiveness 

Support  Staff 

7.4-02 

The  system  shall  portray  the  quantity  of  available 
collection  power  assets  over  time. 

Support  Staff 

7.4-03 

The  system  shall  portray  the  quantity  of  committed 
collection  power  assets  over  time. 

Support  Staff 

7.4-04 

The  system  shall  show  the  amount  of  collection  power 
available  In  contrast  to  the  committed  collection  power 
assets. 

Support  Staff 

7.4-05 

The  system  shall  portray  the  selected  subject's  location 
as  it  relates  to  the  proximity  of  the  available  collection 
power  assets. 

Support  Staff 

7.4-06 

The  system  shall  show  the  trade  offs  between  potential 
collection  power  assets  and  selected  subject  attributes 
options. 

Support  Staff 

7.4-07 

The  system  shall  Indicate  what  current  available 
collection  power  assets  are  deemed  "unusable"  based 
on  environmental  variables. 

Support  Staff 
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7.4-08 

The  system  shall  display  which,  out  of  the  entire  set  of 
Intelligence  Requests,  are  currently  being  collected 
against. 

Commander 

Staff 

7.4-09 

The  system  shall  track  the  total  number  of  collections 
against  each  individual  Intelligence  Request. 

Commander 

Staff 

7.4-10 

The  system  shall  portray  the  life  cycle  (from  planned,  to 
execution,  to  end/return  time)  of  a  collection. 

Support  Staff 

7.4-11 

The  system  shall  provide  the  capability  for  the  user  to 
re-task  a  collection  if  the  subject  attribute  -  collection 
power  pair  is  no  longer  current 

Support  Staff 

7.4-12 

The  system  shall  provide  the  capability  for  the  user  to 
re-task  a  collection  if  the  related  Intelligence  Request's 
priority  has  changed. 

Support  Staff 
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Introduction 


Requirements  elicitations  during  the  Commander’s  Decision  Aid  for  Predictive  Battlespace 
Awareness  (CDA4PBA)  project  uncovered  a  need  for  better  ways  to  request  and  obtain 
information  (see  Section  3.4,  Smart  RFI)  and  better  ways  to  visualize  information  for  effects- 
based  center  of  gravity  (COG)  analysis.  That  effort  led  to  the  Center  of  Gravity  Visualization 
(COG  Viz)  project. 

COG  Viz  project  was  initially  conceived  as  a  preliminary  investigative  effort  to  identify  potential 
tools  and  methods  to  enhance,  in  both  speed  and  accuracy,  center  of  gravity  analysis.  Activities 
conducted  under  COG  Viz  included  a  literature  survey,  COG  tools  and  methods  comparison  and 
gap  analysis,  elicitations  at  the  Warfighter  Analysis  Workshop,  and  baseline  requirements 
development.  The  baseline  requirements  and  analytical  insights  gained  became  the  foundation 
for  a  second,  and  broader,  effort.  Visualization  for  Operational  Environment  Understanding  and 
Response  (VOEUR)  which  is  ongoing  at  the  time  of  this  writing. 


Center  of  Gravity  Visualization  (COG  Viz) 

The  COG  Viz  project  objective  was  to  facilitate  predictive  battlespace  awareness  through 
visualizing  adversary  and  friendly  centers  of  gravity  in  order  to  support  mission  planning 
decision-making  prior  to  and  during  execution.  For  purposes  of  this  effort,  centers  of  gravity 
were  defined  as  “those  characteristics,  capabilities,  or  sources  of  power  from  which  a  military 
force  derives  its  freedom  of  action,  physical  strength,  or  will  to  fight’’  (Joint  Publication  1-02, 
Department  of  Defense  Dictionary  of  Military  and  Associated  Terms).  The  basic  COG  process 
(Figure  1)  is  delineated  in  AFDD  2,  The  Organization  and  Employment  of  Aerospace  Power. 
This  process  was  used  as  a  baseline  for  developing  a  more  in-depth  understanding  of  COG 
visualization  requirements.  Note  the  lack  of  definition  for  how  to  determine  the  COG. 
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Complete  COG  Process 

o  © 


AFDD  2,  Organization  &  Empioyment  of  Aerospace  Power 


Figure  1.  Baseline  COG  process.  From  AFDD  2.  Organization  and  Employment  of 
Aerospace  Power  (2000). 


Current  military  doctrine  identifies  multiple  planning  models;  Figure  2  shows  the  position  of 
COG  analysis  within  three  representative  process  models.  In  each  case,  it  falls  within  the  middle 
of  the  planning  process,  between  orientation  to  the  operational  environment  (OE)  and  actual 
course  of  action  (COA)  development.  It  considers  both  adversarial  capabilities  and  intent  and  is 
part  of  the  conceptualization  of  the  opportunities  and  limitations  of  the  battlespace. 
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The  Joint  air  estimate  process:  A  Sttsopsis 

PHASE  I;  Mission  Analysis 

Conduct  initial  Intelligence  Preparation  of  the  Battlespace  (IPB). 
Phase  I  focuses  on  analyzing  the  joint  force  commander  s  mission 
and  guidance  to  produce  a  joint  air  component  mission  statement 
and  commander' s  intent.  ^ 

PHASE  n;  Situation  and  Course  of  Action  (CO A) 
De\tlopment 

IPB  IS  refined  to  include  adversar>' COj^s-Aaat^^zTadversarv'  and 
finendly  centers  of  gravity  (COG).Tfe^op  multiple  air  COAs  or 
one  air  CO.A  with  significant  branches  and  sequels. 

PHASE  ni:  COA  Analysis 

Friendly  COAs  are  wargamed  agamst  adversarv'  COAs. 

PHASE  IV:  COA  Comparison 

Wargaming  results  are  used  to  compare  COAs  against 
predetermined  cnteria. 

PHASE  V:  COA  Selection 

Decision  bnef  to  joint  force  air  component  commander  (JFACC) 
with  CO.A  recommendation.  JFACC  selects  CO.A. 

PHASE  VI:  Joint  .Air  Oper.\tions  Plan 

DE\'EL0PMENT 

Selected  CO.A  is  developed  into  a  joint  air  operations  plan. 

PROCESS  FOR  STEP  THREE  OF  JOINT  INTELLIGENCE 
PREPARATION  OF  THE  BATTLESPACE 


EVALUATE  THE  ADVERSARV 


1 .  Identify  adversary  centers  of  gravity 

2.  Update  or  create  adversary  models 

3.  Determine  the  current  adversary  situation 

4.  Identify  adversary  capabilities 


Figure  2.  COG  analysis  within  three  planning  concepts:  The  Air  Campaign,  Joint  Air 
Estimate  and  Joint  IPB  processes.  From  the  Air  Campaign  Planning  Handbook  (2000),  the 
Joint  Air  Estimate  Handbook  (2005)  and  JP  2-0.1.3  Tactics,  Techniques  &  Procedures  for 
Joint  IPB  (2000). 


This  effort  reviewed  COGs  within  the  context  of  joint,  multi-service,  and  Air  Force  processes. 
The  following  documents  were  employed  to  understand  current  thinking  with  respect  to  joint 
(and  Air  Force)  planning  and  COG  analysis: 


1.  Air  Force  Pamphlet  (AFPAM)  14-118,  Aerospace  Intelligence  Preparation  of  the 
Battlespace  (2001) 

2.  Joint  Publication  (JP)  2-01.3  Joint  Tactics,  Techniques,  and  Procedures  for  Joint 
Intelligence  Preparation  of  the  Battlespace  (2000) 

3.  JP  3-0,  Joint  Operations,  Revised  Second  Draft  (2005)  and  JP  3-0,  Joint  Operations 

(2001) 
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4.  Air  Force  Doctrine  Document  (AFDD)  2,  Operations  and  Organization(2006)  and 
AFDD  2,  Organization  and  Employment  of  Aerospace  Power  (2000) 

5.  Joint  Air  Estimate  Planning  Handbook,  F.  5  (2005) 

6.  Commander ’s  Handbook  for  an  Effects-Based  Approach  to  Joint  Operations  (2006) 

7.  Supplement  One  to  Commander’s  Handbook  for  an  Effects-Based  Approach  to  Joint 
Operations  (Theory)  (2006) 

8.  Supplement  Two  to  Commander’s  Handbook  for  an  Effects-Based  Approach  to  Joint 
Operations  (Operational  Net  Assessment)  (2006) 

9.  Joint  Forces  Staff  College  (JFSC)  Pub  1,  Joint  Staff  Officers  ’  Guide  (2000) 

10.  Air  Campaign  Planning  Handbook  (2000) 

The  earlier  doctrine  suggests  nodal  analysis  employing  Warden’s  Strategic  Ring  model — 
(Leadership,  Organic  Essentials,  Infrastructure,  Fielded  Forces,  and  Population  (Fadok,  1995). 
More  recent  works,  especially  effects-based  literature,  advocate  an  updated  version,  Barlow’s 
National  Elements  of  Value  (NEV)  model — Leadership,  Industry,  Armed  Forces,  Population, 
Transportation,  Communications,  Alliances;  these  categories  represent  commonalities  in 
strategic  thinking  from  Clausewitz  through  Warden  (Barlow,  1992).  Barlow’s  model  provides 
additional  sensitivity  by  allowing  for  differences  in  relative  importance  of  nodes  and 
relationships.  Another  model,  perhaps  less  familiar  to  Air  Force  strategists,  was  developed  at  the 
Army  War  College  Center  for  Strategic  Leadership  (AWC  CSL).  This  model  considers 
demographic,  economic,  geographic,  historic,  international,  military,  political  and  psychological 
factors  as  well  as  interests  and  political  goals  (Fowler,  2002).  The  CSL  model  has  been  used  in 
an  ongoing  DARPA-sponsored  project  on  artificial  intelligence-aided  COG  analysis  (Teguci, 
2004). 

The  most  recent  planning  doctrine  represents  an  effects-based  paradigm.  The  Joint  Force 
Command’s  Draft  EBO  Concept  Primer  (2003)  states, 

“Actions  in  a  traditional  plan  are  typically  arranged  around  axes  of  advance  or 
lines  of  march  along  linear  sets  of  decisive  points  that  lead  to  a  defined  center  of 
gravity,  whose  destruction  should  result  in  the  enemy’s  defeat”  (p.  5). 
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The  document  goes  on  to  stress  that  effects-based  planning,  in  focusing  on  the  desired  end  state 
is  less  linear  and  hierarchical;  it  emphasizes  a  dynamic,  integrated  system  of  systems  (SoSA) 
approach  that  transcends  military  action  to  include  diplomatic,  information,  and  economic 
(DIME)  solutions  as  well.  SoSA  models  entities  (including  type  and  criticality),  direct  and 
indirect  relationships  (including  type,  strength  and  criticality),  and  direct  and  indirect  effects  for 
friendly,  adversary,  and  non-aligned  groups.  The  SoSA  model  promises  a  more  complete  and 
accurate  assessment  of  the  adversary,  yielding  a  superior  solution  set. 

While  Warden’s  Rings  and  Barlow’s  NEV  can  be  used  in  effects-based  analysis,  the 
Commander ’s  Handbook  for  an  Effects-Based  Approach  to  Joint  Operations  (2006)  employs  a 
new  battlespace  model;  OE  elements  are  categories  as  Political,  Military,  Economic,  Social, 
Infrastructure  and  Information  (PMESII).  Figure  3  shows  a  notional  PMESII-based  SoSA 
identifying  COGs  and  opportunities  to  apply  DIME  actions. 


SYSTEMS  PERSPECTIVE  OF  THE 
OPERATIONAL  ENVIRONMENT 


JIPB 


SoSA 


CDO  Mrtbar  or  tinvitf 

JlPfl  lolnl  IntDllIgDficv  prafuriElcfi  nl  tuUJHpaco 


OPERATIONAL  NETASSESSMENT 
APPROACH  TO  INFLUENCING  THE 
OPERATIONAL  ENVIRONMENT 


DIME  Action . on . PMESII  Systems 


Figure  3.  Nodal  analysis  from  SoSA  and  ONA  perspectives.  From  the  Commander’s 
Handbook  for  an  Effects-Based  Approach  to  Joint  Operations  and  Commander’s 
Handbook... Supplement  Two  (Operational  Net  Assessment). 


The  models  above  are  simplified  depictions.  The  Commanders  Handbook,  Supplement  One — 
Theory  (2006),  clarifies  the  scope  of  a  complete  SoSA  as  envisioned  in  current  EBO  doctrine: 


57 


“Unlike  Clausewitzian  thought,  an  effects-based  approach  extends  beyond  the 
enemy  to  the  entire  OE  and  its  political,  economic,  social,  ideological  and  other 
enabling  systems  that  support  the  global,  regional,  or  national  grouping  to  be 
influenced.  These  systems  may  be  trans-regional,  transnational,  or  connected  in 
functional  and  behavioral  ways  that  are  based  on  political,  familial,  commercial  or 
cultural  relationships.  The  point  is  that  an  effects-based  approach  takes  a 
systems  perspective  to  explain  the  behavior  of  the  entire  OE:  how  it  currently 
behaves  and  how  it  might  behave  under  various  influences  and  actions.”  (p. 

8,  italics  added  for  emphasis) 

Supplement  One  to  the  Commander’s  Handbook  for  an  Effects-Based  Approach  to  Joint 
Operations — Theory  (2006)  also  warns, 

“Depending  on  the  effect  desired,  the  importance  of  an  element  [within  the 
analysis]  will  fluctuate.  And  consequently,  if  the  ends  are  not  understood  at  all 
echelons,  the  presumption  will  be  that  the  classical  (and  often  erroneous)  centers 
of  gravity — leadership,  C2  [command  and  control],  lines  of  communication, 
etc. — are  most  relevant  to  the  success  of  the  operation”  (p.  9). 

These  observations  illustrate  the  difficulty  of  achieving  a  comprehensive  COG  analysis — a 
multi-echelon,  PMESII-based,  comprehensive  analysis  is  a  lot  of  work.  Although  SoSA  is 
clearly  an  effective  method,  is  it  a  cost-effective  method?  It  begs  the  question:  Is  it  even 
possible — given  time  constraints  and  attentional  limitations — in  an  OE?  SoSA’s  promise  of 
integrated,  multi-echelon,  multi-disciplinary,  planning  has  great  potential;  exploring  and 
addressing  such  SoSA  issues  provide  the  framework  for  the  COG  Viz  effort. 


COG  Viz  Literature  Survey 

The  COG  Viz  literature  survey  reviewed  documents  from  multiple  disciplines,  spanning  Center 
of  Gravity  (COG)  Theory  and  Application,  Analysis  and  Modeling,  Intelligence  Preparation  of 
the  Battlespace  (IPB),  Operational  Net  Assessment  (ONA),  Effects-Based  Operations  (EBO), 
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and  Planning  Doctrine  and  encompassing  descriptive  and  explanatory  texts  as  well  as  apologetics 
and  critiques.  Sources  included  doctrinal  joint  and  service  publications,  service-sponsored 
opinion  papers,  multi-service  advanced  school  of  military  studies  papers,  military  handbooks  and 
manuals,  instructional  materials,  articles  from  military  professional  journals,  and  think  tank 
evaluative  reports.  The  highlights  of  the  literature  surveys  are  captured  in  Table  1 : 


Table  1.  Center  of  Gravity  Issues  Identified  in  the  COG  Viz  Literature  Survey. 


Issue 

Sources 

The  Joint  Air  Estimate  Planning  process 
places  COG  analysis  within  Phase  II: 
Situation  And  COA  Development  Tasks 
(immediately  following  IPB  refinement) 
after  Phase  I:  Mission  Analysis  and  prior 
to  Phase  III:  COA  Analysis 

USAF.  (2005).  Joint  Air  Estimate  Planning 
Handbook,  V.  5.  Maxwell  AFB,  AL:  Warfare 
Studies  Institute,  College  of  Aerospace  Doctrine, 
Research  and  Education.  [Textbook  for  the  Joint 
Air  Operations  Planning  Course] 

The  Air  IPB  (AIPB)  and  Joint  IPB  (JIPB) 
cycles  place  COG  analysis  within  Step  3: 
Evaluate  the  Adversary  (immediately 
following  Step  2:  Describe  the 

Battlespace’s  Effects  and  prior  to  Step  4: 
Determine  Adversary  COA) 

JCS.  (2000).  JP  2-01.3,  Joint  Tactics,  Techniques, 
and  Procedures  for  Joint  Intelligence  Preparation 
of  the  Battlespace.  Washington,  DC:  Joint  Chiefs 
of  Staff 

USAF.  (2001).  AFPAM  14-118,  Aerospace 
Intelligence  Preparation  of  the  Battlespace. 
Langley  AFB,  VA:  HQ  ACC/INXX 

The  Air  Campaign  Planning  process  places 
COG  analysis  in  Stage  III:  Center  of 
Gravity  Identification  (immediately 

following  Stage  I:  Operational 

Environment  Research  and  Stage  III: 
Objective  Determination) 

USAF.  (2000).  Air  Campaign  Planning 

Handbook.  Maxwell,  AFB,  AL:  Warfare  Studies 
Institute,  College  of  Aerospace  Doctrine, 
Research,  and  Education. 

There  are  multiple  and  conflicting 
definitions  for  the  term  COG  (e.g.,  balance 
point  vs.  strength) 

Echevarria,  A.  (2004).  Center  of  Gravity 

Recommendations  for  Joint  Doctrine.  Joint  Force 
Quarterly,  35,  10-17. 

Johnson,  M.  (2001).  Strange  Gravity:  Toward  a 
Unified  Theory  of  Joint  Warfighting.  Ft 

Leavenworth,  KS:  USA  Command  and  General 
Staff  College 
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There  is  confusion  over  what  constitutes  a 
COG  (e.g.,  military  power  vs.  will  of  the 
people;  land  forces  vs.  air  forces) 

Eikmeier,  D.  (2004).  Center  of  Gravity  Analysis. 
Military  Review,  July- August,  2-5. 

Anderson,  W.  (2004).  Where  You  Sit  and  Centers 
of  Gravity:  Bridging  the  Gap  Between  Army  and 
Air  Force.  Newport,  RI:  Naval  War  College 

There  is  insufficient  instruction  on  how  to 
apply  COG  assessment  models 

Fowler,  C.  (2002).  Center  of  Gravity — Still 
Relevant  After  All  These  Years.  DTIC  #  ADA 
401889.  Carlisle  Barracks,  PA:  US  Army  War 
College. 

Johnson,  M.  (2001).  Strange  Gravity:  Toward  a 
Unified  Theory  of  Joint  Warfighting.  Ft 

Leavenworth,  KS:  USA  Command  and  General 
Staff  College 

Services  disagree  over  whether  an 
adversary  has  single  or  multiple  COGs 

Fowler,  C.  (2002).  Center  of  Gravity — Still 
Relevant  After  All  These  Years.  DTIC  #  ADA 
401889.  Carlisle  Barracks,  PA:  US  Army  War 
College. 

There  is  disagreement  over  whether  COGs 
can  be  found  at  multiple  levels  or  whether  a 
COG  should  pertain  to  the  system  as  a 
whole. 

Echevarria,  A.  (2004).  Center  of  Gravity 

Recommendations  for  Joint  Doctrine.  Joint  Force 
Quarterly,  35,  10-17 

There  are  multiple  methods  for  COGs 
assessment  but  no  guidance  on  whether  or 
when  one  is  preferred  over  another 

Johnson,  M.  (2001).  Strange  Gravity:  Toward  a 
Unified  Theory  of  Joint  Warfighting.  Ft 

Leavenworth,  KS:  USA  Command  and  General 
Staff  College 

Four  models — Warden’s  Five  Strategic 
Rings,  Barlow’s  National  Elements  of 
Value,  the  CSL  model,  and  SoSA’s 
PMESII  construct — require  the 

analyst/planner  to  categorize  the  IPB 

Warden,  J.  (1995).  The  Enemy  as  a  System.  Air 
Power  Journal,  Spring. 

Barlow,  J.  (1995).  Strategic  Paralysis:  An 

Airpower  Theory  for  the  Present.  Maxwell  AFB, 
AL:  Air  University  Press 

Fowler,  C.  (2002).  Center  of  Gravity  Analysis — 
Still  Relevant  After  All  These  Years.  Carlisle 
Barracks,  PA:  Army  War  College 

USAF.  (2006).  Commander’s  Handbook  for  an 
Effects-Based  Approach  to  Joint  Operations. 
Washington,  DC:  HQ  USAF 
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COGs  should  be  strategic  only 

Echevarria,  A.  (2004).  Center  of  Gravity 

Recommendations  for  Joint  Doctrine.  Joint  Force 
Quarterly,  35,  10-17. 

COGs  can  be  strategic  or  operational 

Eikmeier,  D.  (2004).  Center  of  Gravity  Analysis. 
Military  Review,  July- August,  2-5. 

COGs  can  be  strategic,  operational,  or 
tactical 

Vego,  M.  (2000).  Center  of  Gravity  Military 
Review,  March-April,  23-29 

Strange,  J.  &  Irons,  R.  (2004).  Center  of  Gravity: 
What  Clausewitz  Really  Meant.  Joint  Force 
Quarterly,  35,  20-27. 

COGs  are  dynamic,  shifting  over  time  as 
circumstances  change 

Strange,  J.  &  Irons,  R.  (2004).  Center  of  Gravity: 
What  Clausewitz  Really  Meant.  Joint  Force 
Quarterly,  35,  20-27. 

COGs  are  of  questionable  value  to  military 
planning 

Cancian,  M.  (1998)  Centers  of  Gravity  Are  a 
Myth.  Naval  Institute  Proceedings  Magazine, 
September. 

The  information  acquired  during  the  literature  survey  was  used  to  guide  the  tools  and  methods 
review  and  to  develop  the  elicitation  plan.  Specific  insights  were  also  incorporated  into  the 
baseline  requirements.  As  is  evident  from  the  citations  in  Table  1,  the  literature  review  is  rife 
with  contradictions,  admissions  of  confusion,  and  laments  for  lack  of  COG  identification 
methodology — although  the  literature  search  did  turn  up  one  comprehensive  structured  method 
developed  by  the  CSL  (Giles  &  Galvin,  1996).  The  Critical  Capabilities-Critical  Requirements- 
Critical  Vulnerabilities  (CC-CR-CV)  method,  developed  by  Dr.  Joseph  Strange  (aka  the  Strange 
model)  and  taught  at  all  military  schools,  is  a  pragmatic  and  easily  understood  way  to  assess 
COGs.  However,  it  appears  to  assume  that  the  analyst/planner  can  correctly  determine  the  COGs 
in  the  first  place  (Strange  &  Irons,  2004).  History  shows  that  our  efforts  have  not  always  been 
accurate  (Cancian,  1998)  and  the  inclusion  of  non-state  actors  as  adversaries,  necessitated  by  the 
Global  War  on  Terror  (GWOT),  adds  both  ambiguity  and  complexity  to  the  task  (Anderson, 
2004;  Echevarria,  2004;  Strange  &  Irons,  2004).  The  implication  for  a  COG  Visualization  tool  is 
that  flexibility  is  required  to  provide  visualizations  that  support  both  more  and  less  structured 
approaches  to  COG  identification  for  both  state  and  non-state  adversaries. 
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Each  of  the  three  most-eited  categorical  models  (Warden’s  Rings,  Barlow’s  NEV,  PMESll)  has 
its  proponents — and  its  detractors.  Although  services  vary  in  their  interpretation  of  how  to 
implement  Dr.  Joseph  Strange’s  eoneepts,  all  of  the  serviees  teach  the  Strange  model  for  COG 
decomposition.  Thus,  the  guidanee  derived  from  the  literature  search  determined  that,  in  order  to 
be  flexible  enough  for  joint  use  and  to  serve  the  maximum  range  of  users,  the  COG  visualization 
tool  should  aecommodate  different  sehools  of  thought.  Rather  than  supporting  a  single  or 
several  favored  models,  the  tool  should  support  user-defined  models.  It  should  also  feature  a 
taxonomic  structure  that  would  recognize  coneeptual  relationships  across  models.  These 
decisions,  which  sidestepped  ehoosing  among  faetions,  beeame  baseline  requirements  for  the 
COG  Viz  system. 


COG  Tools  &  Methods 

The  seareh  for  tools  used  for  COG  analysis  did  not  turn  up  many  systems.  The  review  initially 
foeused  on  Air  Foree  and  DoD  efforts  but  expanded  to  ineorporate  relevant  tool  and  methods 
efforts  in  other  fields,  most  notably  biology.  The  review  included  a  gap  analysis  that  led  to 
further  identification  of  COG  Viz  requirements. 

1 .  Air  Operations  Center  (AOC)  planning  tools,  the  Theater  Battle  Management  Core  Systems 
(TBMCS)  architecture  and  Information  Warfare  Planning  Capability  (IWPC)  4.2  include  tools 
for  mission  analysis,  course  of  aetion  (CO A)  development,  and  COA  evaluation.  COG  analysis  is 
supported  by  AFRL/IF’s  Strategy  Development  Tool  (SDT),  which  includes  the  COG  Articulator 
component  for  modeling  COGs  (Caroli,  Fayette,  Koziarz  &  Stedman,  2004).  COG  Artieulator 
both  faeilitates  eonstruction  of  “light-weight”  COG  models  that  charaeterize  adversarial 
capabilities  and  models  the  effeet  of  proposed  “interventions”  on  end  state  attainment.  The  eausal 
chains  for  selected  plans  of  action  can  be  exported  from  the  COG  Artieulator  into  the  Causal 
Analysis  Tool  (CAT)  for  probability  assessment  or  direetly  into  the  SDT  plan  editor.  SDT 
Endstate  integrates  Order  of  Battle  (OB)  information,  COG  models,  and  specified  desired  end 
states  to  produce  EBO-focused  strike  target  lists.  Figure  4  shows  the  antieipated  interaetion  of 
SDT  eomponents  (James  &  Daniel,  2005). 
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Figure  4.  Strategy  Development  Tool  (from  Caroli,  Fayette,  Koziarz  &  Stedman,  2004). 

SDT  is  developed  as  a  proposed  AOC  planning  component  with  the  TBMCS  structure;  as  such, 
it  interacts  with  IWPC.  Planning  for  IWPC’s  successor,  Information  Operations  Planning 
Capability-Experimental  version  (lOPC-X)  has  just  begun  this  year.  Lessons  learned  from 
lOPC-X  will  be  incorporated  into  the  joint  tool,  lOPC-J,  which  is  not  expected  to  begin 
development  until  2008.  To  date,  no  firm  decisions  have  been  made  about  which  tools  will 
transition  from  IWPC  to  lOPC.  As  a  component  of  the  Commander’s  Predictive  Environment 
(CPE)  program,  COG  Viz  project  will  feed  development  of  analytical  capabilities  in  these  AOC 
planning  systems. 

2.  The  Sensor  Harvest  program,  developed  by  the  Air  Force  Information  Warfare  Center 
(AFIWC),  is  a  Command  and  Control  Warfare  (C2W)  planning  tool  with  Information  Warfare 
(IW)  capabilities  (Waterman,  2004).  It  supports  both  strategic  and  operational  planning  and  is 
intended  for  use  in  both  deliberate  and  crisis  action  planning  scenarios  (Air  Intelligence  Agency, 
1997).  The  Sensor  Harvest  office  uses  nodal  analysis  to  identify  centers  of  gravity  for  target 
countries,  terrorist  groups,  and  non-governmental  organizations.  Each  center  of  gravity  is 
decomposed  into  critical  nodes  (e.g.,  leadership,  military,  infrastructure,  facilities,  economics, 
politics,  and  culture).  The  analyses  support  development  of  information  warfare  target  files 
which  include  assessments  of  target  node  criticality,  vulnerability,  and  feasibility  in  the  areas  of 
psychological  operations,  deception,  physical  destruction,  electronic  warfare  and  operation 
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security.  Sensor  Harvest  products  are  available  on  SIPRNET.  The  technologies  and  methods 
used  in  Sensor  Harvest  were  not  available  for  inclusion  in  this  report. 

3.  George  Mason  University,  AFRL/IF,  and  the  Army  War  College  Center  for  Strategic 
Leadership  (CSL)  collaborated  on  a  Defense  Advanced  Research  Projects  Agency  (DARPA) 
project — Disciple  Rapid  Knowledge  Formation  and  Reasoning  (RKF),  whose  first  test  case 
involved  COG  identification.  An  artificial  intelligence  research  project,  Disciple  RKF  employs 
elicitation  formats,  ontologies,  rule  sets,  and  intelligent  agents  (Tecuci,  2004).  Its  artificial 
intelligence  research  objective  was  the  development  of  knowledge  bases  and  agents  by  subject 
matter  experts,  with  minimal  assistance  from  knowledge  engineers.  A  concurrent  military 
strategy  research  objective  was  the  development  of  a  systematic  approach  to  center  of  gravity 
determination.  Customized  versions  of  the  Disciple-RKF/COG  agent  have  been  used  in  the 
Army  War  College’s  “Case  Studies  in  Center  of  Gravity  Analysis”  course,  assisting  students  in 
COG  determination  since  2001.  In  the  2004  experiment,  intelligent  agents,  trained  by  teams  of 
experts,  successfully  completed  98.15  percent  of  the  reasoning  steps  required  to  correctly  assess 
adversary  critical  capabilities  in  a  notional  war  scenario.  Future  plans  include  enhancing  Natural 
Language  Processing  (NLP)  capabilities,  improved  knowledge  acquisition  techniques,  and  meta¬ 
rule  formation  to  capture  rationales  for  problem-solving  method  selection. 

4.  Automated  Assistance  with  Intelligence  Preparation  of  the  Battlespace  (A2IPB),  an  AFRL/IF 
program,  provides  a  method  to  collect  and  document  analytical  elements  and  to  capture 
analytical  insights.  The  A2IPB  system  provides  some  support  to  COG  analysis:  the  A2IPB 
Version  2.2.0  Draft  Student  Workbook  (Zeltech,  2004),  explains  the  importance  of  COG  analysis 
within  its  discussion  of  the  A2IPB  and  the  Joint  IPB  (JIPB)  process  and  its  online  Help  system 
directs  the  user  to  its  Electronic  Notebook  to  enter  COGs  {A2IPB  Online  Help  and  Functional 
Guidance — A2IPB  version  3.2.0).  However,  the  Workbook  offers  no  examples  of  COG 
identification,  and  in  its  exercises,  prompts  students  to  model  the  adversary’s  tactics  and  to 
determine  high  value  targets  (HVTs)  based  on  criticality  to  predicted  adversarial  COAs.  The 
Electronic  Notebook  is  the  user’s  information  repository;  the  online  Help  system  explains  how  to 
enter  identified  COGs  rather  than  how  to  establish  COG  identification.  To  identify  COGs  the 
user  is  simply  directed  to  review  the  information  obtained  in  JIPB  cycle  Step  1-Defme  the 


64 


Battlespace  Environment  and  Step  2-Deseribe  the  Battlespaee  Effeets.  A2IPB  publishable 
produets  inelude  Named  Areas  of  Interest  (NAIs),  Target  Areas  of  Interest  (TAIs),  Units, 
Individuals,  Infrastructure  Areas  (lAs),  COAs,  Lines  of  Communication  (LOCs),  Individuals, 
NAI  Matrices,  Target  Value  Matrices,  LOC  Matrices,  and  COA  Matrices — ^but  not  COGs. 

A2IPB  link  analysis  capability  is  provided  by  the  Web-Enabled  Timeline  Analysis  System 
(WebTAS),  which  also  supports  timelines,  map  displays,  graphs,  and  tables.  WebTAS  pulls 
information  from  the  A2IPB  database,  and  through  A2IPB’s  integration  with  Broadsword, 
supports  importation  of  data  from  multiple  data  sources.  However,  although  it  explains  how  to 
run  a  Broadsword  queries,  A2IPB  literature  does  not  list  capabilities  to  autofill  data  fields  or  to 
automated  query  generation.  Nor  does  it  appear  to  employ  NLP  or  support  taxonomy-based 
automated  network  creation.  Information  the  analyst/planner  deems  relevant  is  pasted  into 
Electronic  Notebook;  data  critical  to  product  generation  is  manually  entered  or  copied  and  pasted 
into  system  data  fields.  Network  models  must  be  built  manually.  While  A2IPB  displays 
individual  elements  of  information  in  appropriate  formats  (e.g.,  maps,  imagery,  tables,  etc.),  it 
does  not  appear  to  support  the  large-scale,  multi-echelon  network  analysis  required  for  accurate, 
defensible  COG  identification. 

5.  The  Australian  Government’s  Department  of  Defence  Science  and  Technology  Organisation 
(DSTO)  sponsors  the  Centre  of  Gravity  Network  Effects  Tool  (COGNET).  COGNET  models 
decompose  and  prioritize  COG  elements,  organizing  them  hierarchically  to  show  associated 
critical  and  lower  level  capabilities.  COGNET  represents  COGs  as  Bayesian  nets  to  support 
probabilistic  cause  and  effect  assessments  for  proposed  interventions  targeted  anywhere  within 
the  hierarchical  COG  structure  (Falzon  &  Priest,  2004). 

The  tools  and  methods  review  suggests  a  common  theme.  JFCOM’s  Commander’s  Handbook 
for  an  Effects-Based  Approach  to  Joint  Operations  (2006)  suggests  SoSA-based  ONA  as  an 
analytical  method.  Richard  Bullock  (2002),  while  Chief  of  Theater  Operations  Modeling  at  the 
Air  Force  Studies  and  Analyses  Agency,  and  Alexander  Levis  (2004),  former  Chief  Scientist  of 
the  Air  Force,  proposed  modeling  COGs  with  influence  nets — another  name  for  Bayesian 
inference  nets.  As  noted  above,  AFRL/IF  (Evans,  Jones,  Pioch,  Prendergast  &  White,  2004)  and 
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Australia’s  DSTO  (Falzon  &  Priest,  2004)  employ  Bayesian  nets  to  assess  causal  chains  in  COG 
analysis.  Network  analysis  is  a  critical  technique  in  IPB  analysis.  In  the  intelligence 
community,  non-probabilistic  social/communications  network  analyses  are  accomplished  using 
such  links  and  nodes  tools  as  Analyst  Notebook  and  TELSCOPE  (Henry,  2004;  Wood,  2003). 
The  common  theme  in  the  newest  and  most  capable  COG  analysis  tools  is  network  analysis. 

However,  Lt  Commander  Michael  Hannan  (USN)  sounds  a  warning  note  in  his  2005  paper  on 
ONA.  He  cites  Carnegie  Mellon  University’s  Kathleen  Carley,  who,  in  her  work  for  the  Office 
of  Naval  Research  (ONR),  found  that  “the  tools  available  now  cannot  handle  both  [conceptual 
and  computational]  types  of  information  at  a  fidelity  required  by  ONA”.  While  network 
modeling  capabilities  lag  requirements,  rapid  navigation  through  and  comprehension  of  complex, 
multi-level  network  presentations  is  difficult  as  well  (Mukherjea,  Foley  &  Hudson,  1995). 
Efforts  to  improve  navigation  involve  network  overviews,  rule-based  node  filtering,  drag-and- 
drop  network  manipulation  and  manipulation  of  3-D  in  2-D  representations  (e.g.,  Klein  & 
Reiterer,  2005;  Quang  &  Mao,  2005;  Thinkmap  a  &  b,  2005). 

Visualization  of  the  OE  Common  Operational  Picture  (COP),  a  goal  delineated  in  Joint  Vision 
2020,  is  the  focus  of  multiple  DoD  projects  (e.g.,  programs  by  Analytical  Graphics,  Inc.,  ESRI, 
General  Dynamics,  Microsoft,  SRA  International  Inc.,  etc.).  Yet  most  of  the  efforts  to  produce  a 
COP  are  limited  to  attempts  to  provide  real-time  updates  to  fused  geospatial  and  other  (e.g., 
event,  asset,  demographic,  etc.)  data  sets  (ESRI  Developer’s  Summit,  2006).  The  visualizations 
that  illustrate  how  COGs  impact  the  operational  environment  are  typically  presented  either  in 
matrices  or  in  nodal  displays  that  are  disconnected  from  geospatial  renderings,  making  cognitive 
aiding  through  enhanced  visualization  an  incompletely  achieved  objective.  Work  at  Iowa  State 
in  navigating  complex  metabolic  network  provides  one  potential  solution — 3-D  immersion 
moderated  by  a  3-D  in  2-D  navigation  display  (Dickerson,  Yang,  Blom,  Reinot,  Lie,  Cruz-Neira, 
&  Wurtele,  2004).  The  user  moves  through  the  virtual  network  to  investigate  its  contents  but 
maintains  an  overview,  as  well  as  navigational  control,  through  a  separate,  handheld  version  of 
the  display.  The  same  kind  of  filtering  algorithms  that  support  navigation  can  also  break  the 
network  into  comprehensible  chunks. 
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In  summary,  COG  visualization  requirements  include  interaction  with  TBMCS,  large-scale 
multi-echelon  influence  network  displays,  probabilistic  causal  analysis  capabilities,  data  fusion 
visualizations  that  link  influence  network  data  to  geospatial  and  temporal  data,  fdtering 
capabilities,  rapid  navigation  techniques,  and  both  god’s  eye  and  immersive  perspective. 


Warfighter  Analysis  Workshop  Elicitations 

Elicitations  for  the  COG  Viz  program  were  conducted  both  at  the  preparatory  conference  for  the 
Warfighter  Analysis  Workshop  (pre-WAW)  and  at  SRA.  Comments  were  obtained  from  a  range 
of  planners,  whose  experience  base  included  both  planning  and  assessment.  Interviews  were 
conducted  using  Cognitive  Task  Analysis  procedures,  including  the  Critical  Decision  Method 
(CDM)  interview  technique  developed  by  Klein  Associates,  (Klein,  Calderwood,  &  MacGregor, 
1989).  An  elicitation  probe  document  was  developed  to  guide  elicitations.  Interview  results  were 
analyzed  for  information  requirements  and  decision  points  using  event  sequence  modeling  and 
decision  requirements  tables  (Pyy  &  Andersson,  1997). 

Air  Force  input  was  represented  by  2  active  duty  Lieutenant  Colonels,  2  active  duty  majors,  2 
active  duty  Lieutenants  and  a  DoD  employee.  A  separate  intensive  2-day  interview  was 
conducted  at  SRA  International,  Inc.,  in  order  to  capture  the  planning  expertise  of  a  MAJCOM 
planner.  The  notes  from  these  interviews  are  found  in  Appendix  A.  The  interview  notes  and 
derived  requirements  were  reviewed  by  a  retired  Lieutenant  Colonel  with  operational  expertise  to 
ensure  system  requirements  met  operational  needs.  Insights  derived  from  the  elicitations  are 
found  in  Table  2. 


Table  2.  Elicitation  Insights. 


Observation 

Source 

COG  Analvsis 

System  needs  to  support  integrating  Blue  COG  analysis  and  Red 
COG  analysis  for  integrated  planning 

WAW  Interview  Notes 

System  needs  to  take  COG  analysis  to  target  systems  analysis  and 
to  weapon/target  pairing 

WAW  Interview  Notes 
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Users  need  more  training  in  COG  analysis  and  COG  analysis  tools, 
techniques  and  methods 

WAW  Interview  Notes 

System  needs  to  handle  both  concrete  and  conceptual  COGs  (e.g., 
physical  capabilities  vs.  moral/morale) 

WAW  Interview  Notes 

System  needs  to  lighten  COG  analysis  workload/shorten  COG 
analysis  timeline 

WAW  Interview  Notes 

Which  COG  models  are  used,  and  whether  they  are  used  at  all,  is 
dependent  upon  the  preferences  of  COCOM  OR  JFACC 
commander  (depending  upon  whether  the  planning  situation  is 
during  peace  or  war) 

WAW  Interview  Notes 

System  needs  to  accommodate  multiple  modeling  methods 

WAW  Interview  Notes 

System  should  support  identification  of  1)  Known  Knowns,  2) 
Known  Unknowns,  and  3)  Unknown  Unknowns 

MAJCOM  Notes 

System  should  support  user-developed  models  (e.g.,  people,  stuff, 
money) 

Planning  Integration 

WAW  Interview  Notes 

System  needs  to  support  defense  of  plan  elements — planning 
rationale 

WAW  Interview  Notes 

System  needs  to  support  planning  beyond  iust  military  solutions — 
DIME  vs.  M  only 

WAW  Interview  Notes 

System  needs  to  support  anticipated  damage  and  reconstitution 
planning  associated  with  specific  targeting  schemes 

WAW  Interview  Notes 

System  should  support  planning  based  on  tasks  listed  in  mission 
analysis  brief  (e.g.,  Gain  Air  Superiority,  Conduct  Close  Air 

Support,  Support  CFLCC) 

MAJCOM  Notes 

System  should  support  integrating  plan  phasing  and  scheduling  into 
visualizations 

MAJCOM  Notes 

System  should  support  both  deliberate  and  crisis  action  planning 
timelines 

MAJCOM  Notes 

Visualization 

System  needs  to  support  lines  of  effect  visualization 

WAW  Interview  Notes 

System  needs  to  be  able  to  scale  visualization  outward  (overview 
and  inward  (focused,  detailed  view) 

WAW  Interview  Notes 

System  should  support  geospatial  views 

MAJCOM  Notes 

System  should  support  visualizing  dynamic  nature  of  battlespace 
and  how  desired  effects  morph  over  time 

MAJCOM  Notes 

System  should  support  visualizing  PMESII  or  other  model  elements 

MAJCOM  Notes 
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that  will  impact  plan  success 

System  should  support  visualizing  desired  end  state  vs.  current  state 

MAJCOM  Notes 

System  should  support  toggling  multiple  layers  over  foundation 
visualization 

MAJCOM  Notes 

System  should  support  CO  A  Sketch  project  or  include  CO  A  Sketch 
capability 

MAJCOM  Notes 

System  should  work  with  geospatial  coordinates  and  various 
standardized  geospatial  product  levels  of  detail  (e.g.,  1 :2, 000, 000 
vs.  1:1,000,000  vs.  1:50,000) 

MAJCOM  Notes 

System  should  provide  “thinking  aids”  and  not  just  documentation 
of  thought-out  plans 

MAJCOM  Notes 

Generating  Items 

System  needs  to  support  RFI  generation 

WAW  Interview  Notes 

System  needs  to  support  generation  of  PowerPoint  briefings 

WAW  Interview  Notes 

Making  Assessments 

System  needs  to  make  assessment  easier  for  Operations  Assessment 
Team — not  harder 

WAW  Interview  Notes 

Need  to  be  able  to  assess  success  as  soon  possible  in  order  to  replan 
or  divert  assets 

WAW  Interview  Notes 

System  should  support  ongoing  mission  effects  assessments 

MAJCOM  Notes 

System  should  support  prioritization  and  weight  of  effort 
assessments 

MAJCOM  Notes 

System  should  support  Value-Focused  Thinking  assessments  (e.g., 
use  tactical  objective  as  the  value  model;  see  notes  for  discussion)) 

MAJCOM  Notes 

Coordination 

Strategy  Plans  Division  tools  need  to  support  coordination  and 
information  sharing  with  Combat  Plans  and  Intelligence, 
Surveillance,  and  Reconnaissance  Divisions. 

WAW  Interview  Notes 

System  needs  to  support  multiple  analysts  working  a  problem 

WAW  Interview  Notes 

System  should  pull  in  and  work  with  Order  of  Battle  information, 
target  lists,  restricted  target  lists,  no-strike  lists,  prioritized  effects 
lists,  rules  of  engagement/laws  of  armed  combat,  etc. 

MAJCOM  Notes 

System  should  support  the  use  of  MILSTD  2525  symbology  for 
ease  in  communication  joint  service 

MAJCOM  Notes 

System  should  support  identification  and  tracking  of  Key 

Operations  Objectives  List 

MAJCOM  Notes 
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Computational  Issues 

System  should  be  prepared  to  cope  with  large  task  to  objective 
mapping  (e.g.,  JEFX  2004  had  some  12  operational  objectives 
broken  into  some  1700  tactical  tasks) 

MAJCOM  Notes 

System  should  aid  filtering  of  available  DMPIs  [Designated  Mean 
Point  of  Impact>targets]  by  desired  effects  and  maximum 
effectiveness  (e.g.,  JWAC  shows  possible  10,000  targets  vs.  only 
need  to  hit  4,000  targets) 

MAJCOM  Notes 

Baseline  Requirements  Development 

The  doctrine  and  literature  surveys,  tool  and  methods  review,  and  initial  elicitations  formed  the 
groundwork  for  requirements  development.  The  first  analysis  performed  was  a  sensemaking 
attempt  to  relate  the  COG  identification  processes  described  in  the  JAEP  and  the  AIPB  and  JIPB 
documents.  The  result  was  a  concept  map  that  identified  and  related  the  requisite  tasks 
preliminary  to  COA  development  (see  Figure  5). 


Figure  5.  Concept  map  relating  JAEP,  AIPB  and  JIPB  doctrinal  processes  prior  to  COA 
development. 
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The  information  derived  from  this  analysis  supported  the  elieitation  plan  and  baseline 
requirements  development,  including  a  tentative  use  case  for  COG  analysis.  Figure  6  maps  the 
elements  of  the  process  described  by  the  MAJCOM  planner.  Figure  7  integrates  the  COG 
Analysis  requirements  from  multiple  sources.  The  improved  understanding  of  COG 
requirements  will  drive  future  elicitations  for  the  VOEUR  project. 


Figure  6.  Understanding  of  COG  process  gained  from  MAJCOM  planner  interview. 
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Figure  7.  COG  Analysis  Requirements 


Although  there  are  a  number  of  requirements  that  are  identified  from  the  elieitations,  not  all  of 
them  are  direetly  related  to  visualization  needs.  Further,  the  projeet  team  is  acutely  aware  that  it 
has  only  collected  information  from  a  limit  set  of  SMEs.  Therefore,  the  end  product  of  the  COG 
Viz  effort  is  limited  to  a  baseline  set  of  requirements  and  a  baseline  feature  set.  These  products 
are  being  used  to  plan  future  elicitations  under  the  VOEUR  project  and  to  guide  VOEUR 
concept  visualizations  for  presentation  to  potential  users.  Table  3  shows  some  of  the  features 
derived  from  the  requirements  analysis  (for  additional  information  see  Appendix  B). 


Table  3.  Proposed  baseline  COG  Viz  feature  set. 


Feature  Set  Name 

Description 

Process  Goal 

Nodes 

Nodes  (objects)  can  be 
created  with  differing 
properties  and  viewed 
in  3D  space 

Capturing  of  physical  and 
conceptual  elements  for  visual 
thinking 

Node  characteristics 

Nodes  can  be  coded  by: 

Capturing  of  physical  and 
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Color  (Image) 

Size 

Shape 

conceptual  elements  for  visual 
thinking 

Links 

Nodes  can  be  linked 
(related).  This  is  viewed 
with  lines  within  the  3D 
space.  These  lines  can 
be  differing  color  and 
width. 

•  Understand  relationships 
between  elements 

•  Understand  system 
complexity 

•  Understand  relationships 
between  systems 

•  Understand  related 
requirements  between 
elements 

•  Understand  system 
complexity 

•  Understand  relationships 
between  systems 

Link  Characteristics 

Link  lines  can  be 
differing  widths 

•  Show  strengths  of 
relationships  in  both 
directions 

•  How  hard  it  is  to  break 
the  relationship 

Categorization 

Nodes  can  be  laid  out 
(placed)  within  the  3D 
space  on  differing 
planes  in  the  3d  space 
based  on  any  Node 
property. 

•  Grouping  or 
categorization  of  a  set  of 
relationships  that  exist 
only  upon  an  individual 
working  in  the  given 
role?  Indicates  that  each 
role  has  its  own  system 
to  consider. 

•  Grouping  of  data  by  user 
defined  purpose 

Filtering 

Nodes  can  be  filtered 
form  the  3d  view  based 
on  any  node  property  or 
any  relationship 
property 

•  Grouping  or 
categorization  of  a  set  of 
relationships  that  exist 
only  upon  an  individual 
working  in  the  given 
role?  Indicates  that  each 
role  has  its  own  system 
to  consider. 

•  Grouping  of  data  by  user 
defined  purpose 

Annotations 

Flags  or  notes  or 
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annotations  can  be 
placed  onto  any  node 
and  are  displayed  in  the 
3d  space 

First  person  View 

The  3d  space  can  be 
viewed  in  a  first  person 
view 

Third  person  view 

The  3d  space  can  be 
viewed  in  third  person 
view  with  an  avatar  on 
screen  representing 
current  location  within 
3D  space. 

Properties  sub  view 

The  properties  sub  view 
will  display  all  node 
properties  of  the  node 
that  is  selected  in  the  3d 
space 

Past  Flight  path 

A  breadcrumb  like  trail 
can  be  turned  on  within 
the  3D  space  to  give 
user  where  they’ve  been 
information. 

Importance 

Importance  can  be 
represented  by  node 
shape,  size,  color,  and 
spatial  placement. 

•  Establish  differential 
importance  to  assist  in 
narrowing  the  focus  to 
the  key  relationships  in 
the  dataset 

Node  Decomposition 

One  can  “walk  into”  a 
system  represented  by  a 
node  and  see  what  is  in 
that  system 

• 

Interactive  Updates 

•  Event  driven 
update  to  nodes 
relationships 
and  their 
attributes  based 
on  assessment 
of  the 
operational 
environment 

•  Show  history  of 
at  least  prior 
state 

•  Allow  user  to 
set  snapshots  of 

•  Adjustment  of  evaluation 
due  to  changes  in 

Centers  of  Gravity  and 
other  key  nodes  due  to 
actions  taken  and 
reactions  of  the 
adversary 
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state  at  their 
discretion 
•  Track  of  state 
changes  over 
time 

Establish  quantified  importance 
based  on  analysis  of  the 
interdependencies  (subordinate, 
superior,  peer) 

Establish  differential  importance 
to  assist  in  narrowing  the  focus 
to  the  key  elements  in  the 
dataset 

•  Matching  to  Laws  of 
Armed  conflict  and  other 
social  moral  restrictions 
in  order  to  guide 
approved  actions? 

(Allow  for  interpretation 
of  potential  allowed 
targets  otherwise 
restricted  but  due  to 
associations  to  approved 
target  type  are  now  legal) 

•  Matching  to  DIME 
actions  that  can  be 
applied? 

Conclusions 

The  COG  Viz  project  successfully  elicited  and  documented  requirements,  concerns,  issues,  and 
desired  features  for  both  COG  analysis  and  for  OE  analysis  with  respect  to  strategic  and 
operational  planning.  Sources  included  doctrine,  papers  for  advanced  military  studies, 
professional  military  journal  articles,  as  well  as  elicitations  with  potential  users.  Primary 
requirements  include  flexible,  user-defined  modeling  methods,  integration  of  massive  amounts  of 
data,  user-defined  data  filtering,  inferencing  capabilities,  multiple  network  analysis  methods 
(e.g.,  social  nets,  influence  nets,  ecological  nets),  integrated  operational  environment 
visualizations,  detailed  and  synoptic  views,  easy  navigation,  and  planning  aids  that  extend 
beyond  current  plan  documentation  capabilities  into  cognitive  aiding  and  analytical  support.  The 
current  VOEUR  project  continues  to  elicit,  document,  and  assess  these  requirements  and  will 
develop  prototype  demonstrations  for  evaluation  by  the  user  community. 
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Acronym  List 


A2IPB 

AFDD 

AFIWC 

AFP  AM 

AFRL/HE 

AFRL/IF 

AIA 

AIPB 

AOC 

AWC 

C2 

CAT 

CC 

CDA4PBA 

COA 

COG 

COG  Viz 

COGNET 

COP 

CPE 

CR 

CSL 

DIME 

CV 

DoD 

DSTO 

EBO 

HVT 

lA 

lOPC 

lOPC-J 

lOPC-X 

IPB 

IW 

IWPC 

JCS 

JFCOM 

JIPB 

JP 

LOC 

MAJCOM 

NAI 


Automated  Assistance  with  Intelligence  Preparation  of  the  Battlespace 

Air  Force  Doctrine  Document 

Air  Force  Information  Warfare  Center 

Air  Force  Pamphlet 

Air  Force  Research  Laboratory  Human  Effectiveness  Directorate 

Air  Force  Research  Laboratory  /Information  Directorate 

Air  Intelligence  Agency 

Air  Intelligence  Preparation  of  the  Battlespace 

Air  Operations  Center 

Army  War  College 

Command  and  Control 

Causal  Analysis  Tool 

Critical  Capability 

Commander’s  Decision  Aid  for  Predictive  Battlespace  Awareness 

Course  of  Action 

Center  of  Gravity 

Center  of  Gravity  Visualization 

Center  of  Gravity  Network  Effects  Tool 

Common  Operational  Picture 

Commander’s  Predictive  Environment 

Critical  Requirement 

Center  for  Strategic  Leadership 

Diplomatic,  Information,  Military,  Economic 

Critical  Vulnerability 

Department  of  Defense 

DSTO 

Effects-based  Operations 
High  Value  Target 
Infrastructure  Area 

Information  Operations  Planning  Capability 
Information  Operations  Planning  Capability-Joint 
Information  Operations  Planning  Capability-Experimental 
Intelligence  Preparation  of  the  Battlespace 
Information  Warfare 
Information  Warfare  Planning  Capability 
Joint  Chiefs  of  Staff 
Joint  Force  Command 

Joint  Intelligence  Preparation  of  the  Battlespace 

Joint  Publication 

Lines  of  Communication 

Major  Command 

Named  Area  of  Interest 
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NEV 

National  Elements  of  Value 

OB 

Order  of  Battle 

OE 

Operational  Environment 

ONA 

Operational  Net  Assessment 

ONR 

Office  of  Naval  Research 

PMESII 

Political,  Military,  Economic,  Social,  Infrastructure  and  Information 

SDT 

Strategy  Development  Tool 

SIPRNET 

Secret  Internet  Protocol  Router  Network 

SoSA 

System  of  Systems  Analysis 

SRA 

SRA,  International 

TAI 

Target  Area  of  Interest 

TBMCS 

Theater  Battle  Management  Core  System 

USA 

United  States  Army 

USAF 

United  States  Air  Force 

USN 

United  States  Navy 

VOEUR 

Visualization  for  Operational  Understanding  and  Response 

WAW 

Warfighter  Analysis  Workshop 

WebTAS 

Web-Enabled  Time  Analysis  System 
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PRE-WAW Interview  Notes 

Notes  from  Cog  Viz  session  2,  Conference  Room  A,  April  5,  2006  1415 

Government  sponsor  provided  the  high  level  briefing  that  included  only  2  slides.  She  handed  it 
over  to  moderator  lead  for  discussion. 

Within  CO  A  development,  we  write  an  operational  or  tactical  objective  directed  at  the  adversary, 
directed  at  their  COG.  Alternative  COAs  really  depend  on  how  much  we  focus  on  that  COG. 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re  going  to  use?  I’ve  never  heard  of  any 
of  the  others.  You  look  at  the  rings  and  you  hand  them  out  to  different  people.  Divide  them  by 
elements  in  the  model  (leadership,  economics,  etc.)  You  then  decide  what  are  the  COGs  for  the 
target  area. 

USMC  and  Army  are  far  more  rigid  within  the  MDMP  than  we  are. 

Army  Reg.  5.0  is  where  MDMP  can  be  found. 

Air  Force  uses  their  own  process,  which  is  much  like  MDMP.  We  think  we  can’t  issue  Orders, 
but  we  do.  We  think  we  only  issue  plans. 

They  took  tours  of  various  organizations  around  DC  so  they  could  identify  experts  for  various 
world  wide  areas.  They  brought  many  of  them  down  to  where  they  are  to  talk  about  their  areas 
of  expertise. 

Dr.  Kevin  Pollack,  Brookings  Institute,  is  one  of  those  experts. 

It  is  typically  ad  hoc  to  figure  out  where  you  get  your  data. 

In  exercises  they  try  to  go  by  doctrine,  which  is  not  what  they  do  in  real  life.  In  real  life,  the 
personality  of  the  leaders  guides  how  things  are  done. 

Where  do  you  get  info? 

Primary  sources: 

Secret  Service 
State  Department 

You  go  wherever  you  can  to  get  the  information  you  need. 

Another  guy: 

I  found  lots  of  information  digging  through  target  folders. 

How  did  you  access  the  target  folders? 

SIPRNET  back  to  CENTCOM. 

Guys  at  32"‘^  AOG  used  those  folders  also. 
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I  created  my  own  database  for  the  primary  targets. 

Included  various  options,  geographic  data,  targeting  options.  He  plotted  them  on  maps 
using  FalconView  on  ITS. 

“Remember,  that’s  the  way  “named  colleague”  did  it.  It’s  so  personality  dependent.” 

The  mindset  will  be  different  based  on  mission  type.  I  did  narcotics  interdiction.  We  looked  for 
information  to  give  to  the  Columbians  so  that  they  can  handle  the  situation.  My  entire  context  is 
different  because  my  operation  is  non-kinetic.  I  am  in  support  of  SOF  missions. 

A  lot  of  units  use  the  CIA  world  fact  book.  They  break  it  down  into  the  standard  categories. 

That  gives  the  intel  analysts  a  starting  point  to  go  and  find  out  more  about ...  the  government, 
for  example.  They  might  look  at  literacy  rate  to  tailor  the  pamphlets  to  that  level. 

As  an  intel  group,  the  fact  book  gives  us  our  generalization.  We  then  use  that  to  go  get  the  nitty 
gritty. 

Trying  to  identify  critical  capabilities  and  vulnerabilities,  I  use  a  system  of  systems  approach. 

We  need  to  identify  what  they  rely  on  for  various  things,  like  power  or  heat. 

What  is  the  education  level?  You  need  a  way  to  document  these  areas  so  that  if  they  become 
important,  we  see  them.  A  checklist  style  format  might  work. 

Once  you  identify  your  COGs,  and  you  predict  something  based  on  an  event  and  they  do 
something  different,  you  must  reanalyze  your  COGs. 

Difference  between  CO  As  and  branches  and  sequels. 

I  have  alternative  potential  plans,  then  analyze  them  against  the  enemy,  we  make  a 
recommendation.  He  then  chooses  his  preferred  COA.  They  are  blessed  by  the  commander  and 
we’ll  expand  that  and  make  that  the  center  of  the  operational  plan  for  the  JAOC.  COAs  can 
usually  be  preceded  by  the  word  alternative.  Once  we  choose  one,  that’s  the  plan.  When 
building  one,  we  make  assumptions.  These  assumptions  become  the  focal  points  of  the  plan. 

We  generate  alternatives  for  each  assumption. 

We  then  talk  about  what  each  COA  is  trying  to  do.  For  example,  this  COA  will  attack  their 
leadership  and  we  think  this  will  result  in  this  change.  Based  on  that  change,  we  will  then  do  this 
COA. 

Are  you  always  focused  on  the  end  state? 

Yes. 

Do  you  need  to  make  sure  that  a  state  doesn’t  change,  and  therefore  you  need  COGs  to  know 
when  the  situation  might  change  and  you  want  to  change  it  back  to  where  it  was? 
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Operational  COGs  are  different  by  mission  phase.  That  is  true  for  both  Red  and  Blue  forces. 

“I  don’t  find  it  useful  to  have  tactical  COGs.  We  need  to  use  the  COGs  to  help  us  write  COAs, 
effects,  and  objectives.” 

A  COG  is  not  an  Achilles  heel.  The  vulnerability  in  the  strength  is  what  you  can  attack.  We  try 
to  protect  our  own  vulnerabilities.  We  look  for  vulnerabilities  to  exploit  to  defeat  their  forces  or 
we  find  vulnerabilities  in  leadership.  How  do  we  prosecute  those  vulnerabilities. 

You  need  to  continually  update  the  things  that  change  that  might  impact  the  success  of  your 
attack  on  those  vulnerabilities. 

Dr.  Strange  teaches  at  CGSC  at  Quantico. 

When  we  talked  about  the  regime  in  Iraq,  there’s  the  “guy”  and  his  inner  circle.  But  you  had  the 
guy  just  outside  of  that  who  was  really  pulling  all  the  strings. 

If  you  do  enough  research,  the  answer  becomes  obvious. 

The  M  is  going  to  break  out  into  4  areas: 

Air 

Land 

Sea 

SOF 

Warden’s  is  what  they  are  teaching.  But,  we  need  to  dynamically  react  so  people  can  update 
their  buckets. 

They  talk  a  lot  about  Warden  and  then  they  talk  about  flexibility,  “don’t  go  in  with  such  a 
concrete  approach  that  you  can’t  see  the  forest  from  the  trees.” 

Warden  was  a  success  after  Vietnam.  It  is  well  written 

It  is  important  to  pick  a  methodology  and  stick  with  it. 

How  do  we  filter  the  data? 

You  are  automatically  limited. 

If  we  are  covering  the  P.  We  are  limited  as  the  Air  Force.  There’s  the  military  P,  and  within  that 
is  what  the  Air  Force  can  do. 

There  are  parts  of  the  P  problem  that  are  simply  not  my  job,  but  I  must  pay  attention  to  some  of 
those  other  elements. 

Do  you  only  focus  on  the  M? 
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No.  Out  of  the  DIME,  we  are  really  only  the  M.  That  is  all  we  have  to  bear.  Application 
of  the  DIME  means  that  we  are  the  M. 

The  matrix  is:  what  do  I  have  to  bear  in  theater,  matrixed  with  PMESII.  In  almost  any  plan,  you 
can  get  to  a  certain  place  in  lots  of  different  ways. 

How  do  I  show  you  all  of  this? 

Is  it  strict  nodal  analysis? 

Show  nodes  with  most  things  going  to  it? 

Show  nodes  with  the  most  critical  connections? 

How  do  I  make  a  template  to  show  COG  (what’s  that  checklist)? 

A:  It  depends.  Which  makes  the  most  sense  for  certain  situations.  Which  is  the 
easiest  to  understand?  Is  it  personality  based? 

Q:  Should  I  give  the  operator  all  models? 

A:  That  would  confuse  him.  He  will  pick  whatever  the  computer  tells  him. 

I  have  a  problem.  There  is  no  cookie  cutter  solution  for  this.  It  is  too  situationally  dependent. 
Korea  is  different  than  Columbia  which  is  different  than  India. 

A  drop-down  menu  of  Warden,  Strange,  Barlow,  PMESII.  If  you  give  the  operator  this  option, 
will  they  know  which  to  use?  We  can  provide  guidance  regarding  which  to  use. 

“I  will  never  get  on  a  computer  to  get  this  going.  I  grab  creative  people,  get  together,  and 
brainstorm.  To  be  constrained  by  a  computer  program  or  tool  is  not  conducive  to  what  I  try  to  do 
at  the  beginning.” 

How  do  you  do  your  homework  before  those  sessions? 

It  depends  on  the  situation.  I  cannot  give  you  a  model  that  will  apply  to  every  situation. 

These  models  are  ways  to  filter,  they  structure  your  work  into  workable  pieces. 

For  me  to  accomplish  my  mission,  I  need  to  identify  my  end  goal  and  work  backwards.  Example, 
food  following  a  tsunami.  How  do  I  get  enough  food?  How  do  I  protect  these  people?  How  do 
I  protect  them  from  neighbors?  What  do  their  neighbors  have  that  could  threaten  them?  What 
might  I  do  about  those? 

I  could  have  something  really  important  that  has  only  one  relationship.  That  could  be  the  most 
important  node  and  it  has  only  one  pathway. 

I  can  imagine  myself  putting  these  pieces  together,  but  what’s  the  point?  We  need  to  maintain 
focus  on  the  end  state.  If  the  answer  bubbles  to  the  top,  why  waste  time  filling  out  a  model? 

[A  SME  put  up  the  following  graphic.] 
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When  dealing  with  real  world  situations,  you  must  consider  this  a  two  sided  problem. 

Our  actions  are  usually  to  degrade  his  capabilities,  affect  his  will,  oppose  his  actions.  They  are 
trying  to  do  the  same  thing  to  us.  Within  this,  the  COGs  must  be  identified  that  allow  us  to  have 
the  impact  we  want.  COGs  are  described  as  source  of  strength  and  the  will  to  use  that  power. 
COGs  lie  in  an  overlap  between  capabilities  and  will.  Where  does  PMESII  lie?  This  might  lie  in 
the  goals  of  each  side. 

If  I’m  trying  to  analyze  a  COG,  he’s  going  to  use  his  actions  to  oppose  me  (information, 
economic,  diplomatic,  military,  etc.). 

PMESII  conditions  do  not  necessarily  map  to  the  capabilities,  will,  and  actions  of  the  enemy  in  a 
direct  fashion. 

Moderator:  maybe  we  start  to  map  capabilities  and  will  to  COGs.  This  could  provide  an  initial 
starting  point. 

I  like  Wing  Commander  Red  Thompson’s  lines  of  effect.  That  works  for  me.  It  provides  a 
roadmap  for  where  you  might  go. 
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Notes  from  Cog  Viz  session  5,  Conference  Room  B  April  5,  2006  1415  (Interviewer  1) 

COG  -  Understand  the  Battlespaee,  but  no  eonsensus  on  how  to  do  COG  analysis 

COGs  typically  come  down  from  the  JFC  level  (1  person’s  statement  reflecting  COGs  as  handed 
down  from  JFC,  with  no  input;  some  others  had  bi-directional  input  experience) 

Narrowing  the  battlespaee  is  just  Strategy-to-Task 

Ex:  Hurricane  season  is  a  parallel 
Multiple  models 

Likes  Warden’s  simplicity,  but  Warden  doesn’t  provide  much  beyond  that 
Barlow  shows  COG  as  living,  reactive  enemy.  Interactive,  but  hard  to  do 
What  enemy  exists  at  the  end  of  the  day 

Implies  necessity  to  revisit  and  update  COG  analysis  with  information 
changes/augmentations 

Strange  -  breaks  down  into  actionable  chunks  (however,  starts  by  COG  assuming  a  COG 
has  been  identified) 

Wants  to  see  all  three,  because  each  has  certain  capabilities 

If  you  show  them  all,  you  get  multiple  perspectives,  which  may  provide  a  better 
understanding 

Everything  (or  anything)  can  be  used  everywhere  -  there  is  no  standard  methodology  and  choice 
of  method  is  personality-dependent 

Comparison  of  example  COG  at  strategic  vs.  operational  levels 
Strategic  Air  Systems  at  Strategic 
SAMs  (missile  systems)  at  Operational 

Strange  is  used,  but  assumes  COG  already  identified 

Have  JWAC  analysis  (e.g.,  electrical  power)  of  targetable  systems  (and  factors  that  impact 
targetable  systems)  rather  than  COGs 

Geography,  infrastructure,  players,  etc. 

Analytical  foci  map  to  traditional  COGs;  leverage  to  ID  Strange’s  CCs-CVs-CRs 
JFC  presents  COGs  through  00. 

One-to-one  mapping  or  “n”-to-one  mapping?  (we  think  many-to-one  is  more  likely) 

JWAC  does  the  Target  System  Analysis  -  get  recommendation  on  how  to  take  down 
LOC,  infrastructure,  geo,  etc. 

JWAC  does  SoSA,  not  COG  analysis 

JFC-provided  COGs 
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Have  all  00  chain,  end  states,  . . . 

What  I  know  through  intel  mapped  to  what  is  needed 

*PMESII  mapping  is  all  subjective 

Look  at  each  element,  what  do  we  know,  capability,  many  details 
*Huge  human  element,  so  need  to  less  subjectively  and  more  objectively  define  the 
problem 

What  are  potential  consequences  of  PMESII  situations 

Map  PMESII  information  to  OO/COGs/effects  to  aid  analyst  critical  thinking 

Viz  will  not  be  main  focus  (maybe  wizard-based  approach,  e.g..  Turbo  Tax) 

How  to  get  help:  through  templates,  checklists,  guidance  on  how  to  do  analysis 
Pull  from  lessons  learned 

Strategy  Plans  must  revisit:  Known-Knowns,  Known-Unknowns,  Unknown-Unknowns 
VPT-like  system  infers  the  unknown  unknowns  and  flags  analyst  attention 

COG  has  to  accommodate  squishy  as  well  as  firm 
“Will  of  people,”  “Religious  extremism” 

COG  of  Iraq  could  not  be  found 

Barlow  shows  a  live  enemy 

Stressing  dynamic  nature  of  situation — ^tie  to  visualization  requirements. 

Investigate  what  we  should  be  doing 

Analytical  capability  depends  on  #  of  augmentees  (airmen  not  normally  in  the  position),  etc. 
Level  of  expertise  may  vary. . . 

COG  could  help  jump  start  RFI  process 

Federated  (aggregated)  population  of  data  (controlled  by  ISRD) 

Should  control  population  of  data  structure 
Make  COG  analysis  federated  as  well 

High  level  of  expertise  is  available  through  reachback  to  Checkmate 

Airmen  have  conducted  COG  analysis 

Many  times  may  not  agree  with  Strategic  view 

Depends  on  COCOM  if  the  COG  is  handed  down  or  if  SPD  is  participatory  in  its 
identification 

Baggage  associated  with  COG,  not  so  with  TSA 

JDAMs  will  complicate  analysis,  because  targeting  can  be  more  specific  and  can  be  many  more 
potential  targets  suggested 
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Requires  collaborative  work 

Layers  should  be  fully  interactive  to  accept  inputs  from  individual  tablets 
Should  be  able  to  update 
main  display 
individual  views 

L*  pass  to  understand  adversary  and  prioritized  unconstrained 
Targets:  should  ->  like  ->  can? 

Done  with  people  sitting  around  table 

Getting  to  Weapon-Target  pairing  (but  at  highest  level) 

Could  we  do  with  DIME? 

Level  of  detail  is  to  bomb,  but  not  what  to  bomb 

Example:  Request  military  action  but  need  flyover  or  basing  rights,  which 
requires  a  request  for  diplomatic  action 

Needs  to  occur  in  training  prior  to  AOC  (from  ISRD  rep  in  SPD) 

Do  COG  analysis  training  at  ISRD,  IWFs,  etc. 

Many  young  airmen  don’t  have  the  critical  thinking  skills 

IF  doing  the  work  of  more  experienced  analysts,  must  have  templates,  prompts,  warnings, 
checklists,  etc. 

Structure  aids  so  everyone  can  get  what  they  need  without  forcing  aids  on  those  who 
don’t  need/want  them 

Analysts  populating  database 
Next  analysts  continue  the  analysis 
Continue  on  through  ‘n’  analyst/planners 

With  any  three  analysts,  can  have  information  presented  three  different  ways,  making  it 
difficult  to  recognize  they  are  the  same 
Also  makes  traceability  difficult 

IPB  is  PowerPoint,  with  textual  backup 

Key  component,  trash-in/trash-out 

May  not  get  customer  feedback  into  models  and  results 

Dream  is  a  live  Barlow  model 
Don’t’  care  about  geo 
Compare  CO  As  (what  happened) 

*Fed  by  Assessment 

Real-time  and  screen  grab  to  play  with 

Highlight  back  to  Strategy 
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“What  should  we  target?”  versus  saying  “COG” 

JFACC  may  limit  #  CO  As  or  COGs  to  focus  on 

*Where  am  I  in  plan-execute  (component  can  have  inputs)? 

Joint  approach  may  not  work  (other  components  are  happy  with  their  process) 

May  be  overlay  each  component 
Have  to  sell  them  on  ideas 

*Really  needs  to  go  beyond  military  (DIME) 

Diplomatic  example  for  AF 

Overflight,  e.g.,  Turkey 

After  9/11,  reachback  contributions  came  in,  e.g.,  from  Checkmate 
AFFORs  job  is  knowing  basing 

AC  problem:  significant  portion  of  plan  that  must  change,  e.g.,  Turkey 
Diplomatic  approach 

Blue  COG  are  often  ignored  -  need  to  consider  (AFFOR  should  monitor) 

Prioritized  dependent  asset  list  -  PDAL 

Warden  is  good  initial  look  (could  use  Olympic  Rings) 

The  models  do  map  to  each  other 

COG  (models)  is  data  presented  in  different  manners 

“Have  to  stay  focused  on  “your”  mission  (component),  because  JFC  should  oversee” 

Don’t  get  stuck  (peripheral  effects,  but  mission  focus) 

Scale  to  what  “I’m”  looking  at 

Would  like  to  see  examples,  e.g.,  Israel  and  Palestine,  COGs  and  how  these  work  (to  better 
scope) 

Make  sure  OAT  doesn’t  get  overtasked  trying  to  supply  data 

Org  chart  for  PMESII  needs  more  than  lines  (back  brief) 

Economic  -  Army  doesn’t  want  to  wipe  out  infrastructure  and  economic 

TSA  should  have  “reconstitution”  information 
Needed  in  COG 
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Notes  from  Cog  Viz  session  5,  Conference  Room  B  April  5,  2006  1415  (Interviewer  2) 

Trash  in  =  trash  out.  All  comes  down  to  understanding  the  Battlespace 
Strange  starts  with  COG,  and  then  breaks  it  down. 

PMESII  does  same 

Barlow  is  closer  to  reality.  Just  because  it’s  closer  to  reality,  doesn’t  mean  it’s  easier  to 
apply. 

I’d  like  to  see  all  models  perspectives. 

This  will  determine  where  highest  probability  payoff  is. 

Not  only  determine  COGS 

Ever5dhing  is  being  used,  but  Strange  is  most  common.  Different  in  every  AOC.  Depends  on 
Strategy  Chief 

I’ve  never  seen  a  model  of  the  enemy  put  up  on  the  data  wall  in  the  AOC. 

With  this  asset,  how  much  can  I  collect. . .  ? 

How  do  you  handle  politics? 

Q.  Who  determines  where  the  COG  is,  say  in  the  case  of  electrical  power? 

A.  A  complex  target  analysis  should  have  been  done  already. 

Q.  How  is  that  presented  to  the  Air  Component? 

A.  Operational  Objectives.  JWAC  will  do  analysis  and  present  options. 

We  take  a  snapshot  of  the  DIA  MIDB.  What  can  we  do  with  this  JWAC  product? 
They  give  you  different  options  for  how  to  take  it  out,  how  long  it  will  be  down 
for,  etc.  (JWAC  does  SoSA,  not  COG  analysis) 

The  process  is  in  place.  But  there’s  what  should  be,  and  there’s  what  is.  If  you  don’t  get  the 
guidance,  you  make  it  up.  (assumption). 

If  I  take  down  the  power,  how  much  does  this  regime  rely  on  it?  Are  there  other  countries 
reliant  on  it? 

Start  with  political.  Constrain  discussion  on  that  until  done.  Then  we  move  on  to  the  next 
one.  Military.  What  are  their  fielded  forces?  Nighttime  capability?  Currency  of  pilots. 

Don’t  think  you  can  computer  model  everything.  Some  things  are  human-centric. 

Federated  population  -  not  just  Strategy  Plans. 

Nobody  should  be  your  boss’s  filter  but  you! 

Q.  Have  we  done  COG  analysis  since  Desert  Storm? 

A.  Yes,  but  they  didn’t  necessarily  agree. 

There’s  baggage  associated  with  COG  analysis,  but  not  TSA — yet  they  are  very 
similar. 

Target  2000  targets,  strike  2000  targets.  That  was  strategy.  Attrition  was  the 
effect. 
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You’ll  take  your  best  shot  at  understanding  the  enemy  (PMESII).  Then  you’ll  figure  out  what  the 
fastest  way  to  take  him  down  with  the  asset’s  you’ve  got.  The  targeteers  will  be  instrumental  in 
determining  this. 

What  should  we  target? 

What  can  we  target? 

What  do  we  want  to  target? 

Is  on  No-Strike  List?  Do  we  have  the  necessary  weapons?,  etc. 

Bunch  of  guys  sitting  around  a  white  board  do  the  analysis.  Break  it  down  into  bite-sized  pieces. 
Ultimately  getting  to  weapon  target  pairings.  Think  any  type  of  target  (person,  place, 
thing,  etc.).  Think  any  capability  (action). 

Strategy  plans  is  looking  at  the  outcome  of  TSA. 

Primary  vehicle  is  PowerPoint  to  CFACC.  Some  are  250  slides. 

By  and  large,  analysts  don’t  have  the  critical  thinking  skills  needed. 

If  your  IPB  is  bad,  then  all  subsequent  analysis  is  bad  as  well. 

Wardens  Rings  are  useless.  Going  to  be  a  combination  of  multiple  models.  Look  at  enemy  and 
say,  “this  is  sound.” 

Dream:  live  Barlow  model — ^political  pieces  growing  and  shrinking. 

We’re  abysmal  at  taking  air  strategy  and  showing  how  it  contributes  to  the  campaign  strategy. 
Need  to  look  at  supporting  relationships. 

CFACC  wants  someone  else  to  have  looked  at  all  this,  even  if  you’ve  got  a  different 
opinion. 

I’d  be  happy  if  the  airmen  had  a  seat  at  the  CFACC’s  planning  table. 

Q.  Into  which  tool  do  you  want  a  COG  system  integrated? 

A.  I  want  a  single  application  that  goes  out  and  gets  all  the  information  for  me. 

Preplanning  for  OIF 

Once  components  brought  in,  you’ve  got  reach  back. 

Ground  speed  0.  Get  a  topic,  issue,  throw  it  out,  pass  it  up  the  chain. 

Q.  National  military  strategy,  etc.  is  it  out  of  our  area? 

A.  Yeah.  Did  we  do  it?  Yeah. 

You’ve  created  a  plan  that  is  heavily  based  on  flying  out  of  a  certain  air  base. 
Then  you’re  told  you  can’t,  you  can’t  solve  it  yourself.  So  it  goes  up  the  chain. 
Then  it  might  become  a  diplomatic  issue. 

We  have  been  focused  on  the  enemy.  There’s  a  blue  piece  that  has  been  largely  ignored. 

We’ve  got  the  red  Barlow  model  (we  want  to  attack).  And  we’ve  got  the  blue  model  (we 
want  to  protect).  We  ignore  the  blue  model  a  lot. 

Which  model  is  used  is  dependant  upon  the  personality  of  the  person  using  it. 

Some  models  lend  themselves  to  certain  AORS  better  than  others. 
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Don’t  get  wrapped  around  the  crank  about  cross-component  ripple  effect.  Worry  about  your  own 
piece  of  the  mission.  The  JFC  should  be  worried  about  that. 


System  should  deal  with: 

1)  Known  knowns,  2)  known  unknowns,  and  3)  unknown  unknowns. 
Provide  a  real  world  example  of  COG  Viz.  Maybe  Israel  vs.  Palestine. 
Don’t  saddle  the  OA  guys  with  yet  another  deliverable. 

Political  analysis  vs:  Analyst’s  Notebook  vs.  Organizational  chart 
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MAJCOM  Interview  Notes 


I.  COA  Development  Process 

Start  with  map  of  enemy  territory 

Map  needs  to  be  the  1-250000  view 

Outlines  of  cities,  Topographical  data,  Major  highways,  Lat/Long  grid  marks 
Map  where  JOA  (Joint  Operations  Area)  is  -  where  we  have  control 
Begin  categorizing  activities  based  upon  tasks  listed  in  Mission  Analysis  brief  (Gain  AS, 
conduct  CAS,  support  CFLCC) 

Would  like  to  have  a  library  of  actions  that  you  can  drag/drop  from  based  upon 
activity  category  (These  would  be  Tactical  Objectives) 

Would  like  to  also  refer  to  what  other  components  are  doing  (“army  is  moving  forces  here”) 
If  doing  network  warfare,  would  like  a  network  diagram  instead  of  map  (need  for  other  tools) 
-  map  would  give  90%  solution 
MILSTD  2525  symbology  -  GOOD 

Begin  to  draw  on  map  the  “categories”  and  “actions”  (Gain  Air  Superiority) 

Would  like  controls  that  are  easier  than  PowerPoint! 

Want  to  be  able  to  draw  the  circles/squares/blotches/arrows  of  different  line 
thickness 

Would  like  to  be  able  to  produce  a  COA  sketch  for  each  phase  and  AOD 

May  have  need  for  adding  time  slices  to  one  visualization  (AOD  A  achieved 
this  much,  AOD  B  added  this,  AOD  c  added  this  to  AOD  B) 

Would  like  Intel  to  be  able  to  add  data  and  provide  this  instead  of  PowerPoint  slides 
Would  like  to  be  able  to  toggle  bad  guy  layers 
Would  like  to  be  able  to  brief  this  to  Commander 

Briefs  Phase,  AOD  views  as  well  as  individual  00  views 
Believe  this  would  be  the  CC’s  favorite  tool 

Need  to  show  “refined”  view,  not  computer  generated  which  could  be 
choppy. . . 

Would  like  CC  to  be  able  to  see  and  write  thoughts  down  while  looking  at 
briefing 

Allow  him  to  circle  and  interact  with  map 

Allow  him  a  place  to  write  down  comments  (CC  guidance) 

Would  like  to  use  map  to  begin  planning  -  suck  OO/TO  data  into  CPT  from  map 
Would  like  map  view  to  updated  due  to  changes  as  work  products  (JPITL,  MAAP) 
are  created/detailed 

Extract  Operational  Objectives  and  Tactical  Objectives. 

Begin  to  break  down  00s  and  TOs  into  phases  against  time. 

This  process  will  begin  to  weed  out  bad  ideas  and  will  also  add  new  00s  and  TOs 
Some  phases  are  started  not  based  upon  whether  or  not  we  are  ready  for  the  next 
phase,  but  whether  or  not  the  enemy  takes  action 
Phases  are  usually  broken  down  like  this: 

Phase  1  -  deploy  (set  up  time) 

Phase  2  -  seize  the  initiative  (action!) 

Phase  3  -  culminate  (Need  to  have  MES  achieved  by  end  of  this  phase) 
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All  end  states  that  define  success  should  be  achieved  by  the  end  of  this 
phase 

Phase  4  -  clean  up/  transition 

Military  plays  supporting  role  to  diplomatic/economic  end  states 
Phases  are  event  driven,  not  time  driven! 

CC  will  still  set  an  anticipated  date  based  upon  when  effects  should  be 
accomplished 

Phases  could  possibly  go  on  at  same  time 

i.e.,  I  finish  up  AS  early.  I  need  AS  only  to  begin  new  phase  tasks.  I  must  now 
decide  if  I  should  free  up  resources  for  other  current  phase  activities  or  begin 
with  next  phase’s  operations. 

Will  submit  RFIs  throughout  this  process 

If  there  are  multiple  ways  to  achieve  an  effect  -  they  will  do  all  to  ensure  success 
Will  create  new  COAs  by  merging  COA  ideas 

This  is  sometimes  done  by  the  commander 

COA  Development  will  produce:  (believe  a  lot  of  this  could  be  done  in  the  COA  sketch  if 
tool  is  done  right) 

COA  sketch 
Operational  Concept 

Paragraph  of  what  we  are  going  to  do  (the  HOW  that  is  missing  from  mission 
analysis) 

Key  Operations  Objectives  List 
List  of  forces  required 

Allocation  planner  fits  here! 

Current  makes  educated  guesses  by  reverse  engineering  DMPI/sortie  equiv 
May  have  phase  plan 


II.  Effects-based  Operations  —  Value-focused  thinking 

EBO  does  not  tell  you  what  to  do;  it  tells  you  when  to  stop 
End  state  =  adversarial  Status  System  =  Effect 


Planning  model  for  each  chain  of  effects-based  actions: 

No  action  Action 

Effect  Effect 


Actions 


F 

T 

T 

FT 

XT 

F 

FF 

TF 

No  action  Action 

No  effect  No  effect 
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Air  Superiority  example 

Accompanied  by  spreadsheet  with  value  model  (AirSuperiority.xls) 

Air  Superiority  End  State:  Tactical  Objective  (TO) 

Air  Superiority  Defensive  System  includes  SAMs,  AAA,  and  IADS 

1.  Freedom  from  attack  or  other  Red  action  2.  Freedom  for  Blue  action 

Air  Superiority  employs  Offensive  Counterair,  Defensive  Counterair,  and  Base  Defense 

Concerns:  SAMs,  AAA,  IADS 


Value  Model  for  Aircraft 
affected  by  SAMs 

Rating  Score  Criteria 

Importance 

Destroyed 

.5 

Damaged 

.4 

Di??? 

.1 

Value  Model  for  SA2 

Rating  Score  Criteria  &  #  of  launches 

SA2  launches 

Importance 

1 

.5 

0 

Guided  launches 

.95 

Radar 

.80 

2 

3 

5 

Optical 

.20 

5 

10 

Unguided  launches 

.05 

Ballistic 

1.0 

5 

10 

Note:  Observe  difference  in  relative  value — .95  vs.  .05;  (x  •  y  •  z)  =  1  [perfect  score] 
Focus:  Tactical  Task  (TT)  Neutralize  SA2s 
Effect:  No  SA2  shots  at  my  airplanes 
Action:  Tactical  Task  (TT)  Destroy  TEL  radar 


Destroy 

SA2 

•  TEL 

•  Radar 


Actions 

Effects 

FT 

TT 

FF 

TF 

•  Anticipate  approximately  10-20 
TTs. 

•  EBO:  Actions  taken  will  lead  to 
an  effect  through  causal  linkages 

•  Quad  chart  illustrates  possible 
outcomes 
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Neutralize 

SAMs 

E  (n)  effect 

Actions 

FT 

Oi 

a> 

+  W 

FF 

T] 

Air 

Superiority 

E  (n  +  1)  effect 

Actions 

Operational  Assessment: 

OA  guys  don’t  have  a  way  to  measure  effects 

Use  Tactical  Objective  (TO)  as  value  model 

Note:  Can’t  say  0  aircraft  losses  =  Air  Superiority 

Tactical  Task  (TT)  value  differs  from  Air  Superiority  Model  (00  or  TO) 

TT  only  looks  at  SA  2  launches  vs.  TO  looks  at  loss  of  aircraft  or  numbers  of  ground- 
based  and  sea-launched  attacks 

Differentiate  measures  of  performance  from  measures  of  effect 


Air  Superiority 


Effects  Node 


Destroy  SA  2  s 


Bay  of  Pigs  Scenario 

Mission  Effects  Assessment:  Did  my  troops  kill  Castro 

Did  Castro’s  mistress  kill  him  (unrelated  to  my  actions) 

First  is  a  true  action  (causal  link)  and  a  true  effect  (desired  end  state).  Second  is  an  unknown 
actor  doing  an  unknown  action  (false  action)  and  yet  you  see  your  desired  end  state  (true  effect). 

JEFX  2004  had  12  operational  objectives 
1700  Tactical  Tasks 

Our  job  is  to  qualify  and  quantify  goals  and  progress  and  to  use  visualizations  to  present  to 
CFACC.  CFACC  cares  about  the  number  of  aircraft  diverted. 

CFACCs  have  different  risk  tolerances.  Team  makes  assessments  to  present  risk  ratio  for  actions 
to  effects 

SA2s  destroyed.  Goal  is  .8  by  Day  4.  Do  I  fly  on  Day  4  if  effect  is  achieved  by  Day  3? 


I  want  an  allocation  planner. 

How  many  DMPIs  are  there  vs.  my  actions? 
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Note:  Colleague  made  a  9000  target  spreadsheet  for  Iraq  with  individual  DMPIs  and  allocation 
plan 

You  will  have  OB  with  BE  #s. 

DMPI  identification  depends  on  what  you  want. 

If  none,  not  weaponeered. 

MIDB  plus  CAT  codes  against  target  list 

Just-in-time  targeting.  Can’t  do  allocations  at  the  last  minute — ^have  to  look  outward. 

Targeteers  know  IADS  targets,  etc.  with  CAT  codes  have  a  rough  idea  of  capabilities 

III.  Goals 

1 .  Thinking  aids — a  key  to  understand  relationships 

2.  Shorten  the  24-hour  cycle  to  a  continuous  cycle 

JFACC  Planning  Tool  (90s) 

No  one  understood  the  rule  of  thumb  by  which  system  operated  (?) 

Made  guesses  but  didn’t  know  the  business  rules  embedded  in  system  (guessed  3000  DMPIs  vs. 

2  known  DMPIs) 

Need  to  make  business  rules  explicit  when  business  logic  underlies  a  forecast 

8*  Air  Force  has  a  12  hour  production  cycle 
CAP  and  Time  Sensitive  products  (not  a  full  JAOP) 

Want  tools  to  collaborate — ^not  collaborative  tools.  That  is,  put  decisions  into  tools  rather  than 
focus  on  making  tools  so  “they”  can  see  what  “we”  see. 

Design  tool  to  JAEP  to  meet  MAJCOM/COCOM  needs. 

IV.  Mission  Analysis 

Air  Superiority  is  an  enabler,  not  an  effect  in  and  of  itself  Use  as  a  critical  capability  to  degrade 
political  leadership 

Map  shows  Joint  Ops  area  in  bad  guy  country. 

Circles  indicate  areas  where  you  need  air  superiority,  naval  exclusion  zones,  CFLCC  area  of 
operations,  etc.  Critical  Capability  might  be  communication  and  action  might  be  to  jam  towers. 
Tag  circles  as  operational  objectives,  provide  phases  (groups  of  timing) 

Now  use  PowerPoint  slide. 

Want  interactive  picture  with  metadata  attached. 

COA  development  is  not  done  to  the  TT  level.  May  have  TO  but  may  not  have  parent  named  (?) 
Air  component  perspective  good  but  Joint  perspective  better  for  tool.  The  CFC  signs  off  on  your 
plan  anyway  (not  the  JFACC).  An  electronic  JAOP  would  be  much  more  detailed  than  any 
paper  copy  could  be. 

The  paper  copy  is  an  executive  summary — ^but  if  that  is  what  the  CFC  sees,  that  is  all  he  has 
signed  off  on. 
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Does  an  electronic  ATO  foster  too  early  precision  or  over-precision?  Answer:  No. 

In  such  a  tool,  the  data  provided  to  CFC  is  not  operational  data,  but  is  still  very  detailed.  Now  he 
sees  a  unified  whole.  Gantt  chart  in  PEL — would  change  wording  to  represent  his  desired  effects 
but  didn’t  change  effects. 

PEL  item  changes  broke  linkages;  while  the  activity  remained  basically  the  same 
Changes  require  rewriting  of  plan. 

Operational  objectives  don’t  morph  over  time.  Effects  do  morph  over  time. 

Current  focus — priority,  weight  of  effort. 

Time-phased  picture  of  plan  is  much  more  understandable  than  the  Gantt  chart.  Build  slides,  by 
phase,  on  the  map.  Show  end  state  and  families  of  activities  to  achieve  the  end  state 
(autopopulated  by  CPT) 

Map  display  should  be  able  to  show  up  to  four  CO  As  across  four  phases.  They  will  vary  by 
ends,  means,  ways,  and  risk. 

Disconnect  between  Day  10  requirements  and  capabilities 
Am  I  going  to  be  done? 

Can  I  be  done? 

When  must  I  be  done? 

Go  to  JWAC.  There  are  10,000  DMPIs  but  we  only  need  to  hit  4,000.  What  to  do  to  decrease 
them. 

COA  Sketch:  Operational  Concept  (word  picture  plus  time  flow  picture) 

Then  we  flesh  out  the  details  and  personalize  to  the  bad  guy  country. 

If  one  were  to  make  that  sort  of  tool,  what  sort  of  fidelity  would  be  required  in  the  map  picture. 
JNC  (1:  250,000)  vs.  ONC  vs.  TPC 

Need  outlines  of  countries,  cities,  highways. 

Deliberate  Planning: 

Mission  Analysis-days  to  weeks 

COA  Development-days  to  weeks  to  months  (Time-sensitive  planning  has  short!  timeline) 

COA  Selection-days  to  years  (deliberate  planning  can  take  years  for  approvals) 
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Notes  Acronym  List 


AAA 

AC 

AF 

AFFOR 

AOC 

AOD 

AOG 

AOR 

AS 

ATO 

CAS 

CAT 

CC 

CC 

CENTCOM 

CFACC 

CFC 

CFLCC 

CGSC 

CIA 

COA 

COCOM 

COG 

CPT 

CR 

CV 

DIA 

DIME 

DMPI 

EBO 

F 

IADS 

ISRD 

ITS 

IWF 

JAEP 

JAOC 

JAOP 

JDAM 

JFACC 

JFC 

INC 

JOA 


Anti-Aircraft  Attack 
Aircraft 
Air  Force 
Air  Force  Forees 
Air  Operations  Center 
Air  Operations  Directive 
Air  Operations  Group 
Area  of  Responsibility 
Air  Superiority 
Air  Tasking  Order 
Close  Air  Support 
Causal  Analysis  Tool 
Combatant  Commander 
Critical  Capability 
Central  Command 

Combined  Foree  Air  Component  Commander 

Combined  Foree  Commander 

Combined  Foree  Land  Component  Commander 

Command  &  General  Staff  College 

Central  Intelligenee  Agency 

Course  of  Action 

Combatant  Command 

Center  of  Gravity 

Collaborative  Planning  Tool 

Critieal  Requirement 

Critical  Vulnerability 

Defense  Intelligenee  Ageney 

Diplomatic,  Information,  Military,  Economic 

Designated  Mean  Point  of  Impact 

Effects-based  Operations 

False 

Integrated  Air  Defense  System 

Intelligence,  Surveillance,  Reconnaissanee  Division 

Information  Teehnology  Systems 

Information  Warfare  Flight 

Joint  Air  Estimate  Proeess 

Joint  Air  Operations  Center 

Joint  Air  Operations  Plan 

Joint  Direct  Attack  Munition 

Joint  Force  Air  Component  Commander 

Joint  Foree  Commander 

Jet  Navigation  Chart 

Joint  Operations  Area 
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JPTL 

Joint  Prioritized  Target  List 

JWAC 

Joint  Warfighting  Analysis  Center 

M 

Military 

MAAP 

Master  Air  Attack  Plan 

MAJCOM 

Major  Command 

MDMP 

Military  Decision  Making  Process 

MIDB 

Modernized  Integrated  Data  Base 

MILSTD 

Military  Standard 

OA 

Operations  Assessment 

OAT 

Operations  Assessment  Team 

ONC 

Operational  Navigation  Chart 

00 

Operational  Objective 

P 

Political 

PDAL 

Prioritized  Defended  Asset  List 

PEL 

Prioritized  Effects  List 

PMESII 

Political,  Military,  Economic,  Infrastructure  &  Information 

RFI 

Request  for  Information 

SAM 

Surface-to-Air  Missile 

SME 

Subject  Matter  Expert 

SOF 

Special  Operations  Forces 

SoSA 

System  of  Systems  Analysis 

SPD 

Strategy  Plans  Division 

T 

True 

TEL 

Transporter  Erector  Launcher 

TO 

Tactical  Objective 

TPC 

Tactical  Pilot  Chart 

TSA 

Target  Systems  Analysis 

TT 

Tactical  Task 

USMC 

United  States  Marine  Corps 

VPT 

Visual  Planning  Tool 
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Appendix  B.  Baseline  Requirements 
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COG  Visualization  Requirements  Identified  During  pre-WAW 


ID 

Requirement 

Supporting  Text 

1.0 

COG  Visualization  must  support 
battlespace  understanding 

All  comes  down  to  understanding  the  Battlespace 

1.1 

COG  Visualization  must  support  multiple 
existing  and  user-defined  approaches  for 
COG  Analysis  against  same  data  set 
(Strange,  Barlow,  Warden’s,  SoSA,  etc.) 

Strange  starts  with  COG,  and  then  breaks  it  down. 

PMESII  does  same. 

Barlow  is  closer  to  reality.  Just  because  it’s  closer  to  reality, 
doesn’t  mean  it’s  easier  to  apply. 

I’d  like  to  see  all  models  perspectives. 

Have  we  done  COG  analysis  since  Desert  Storm?  -yes,  but 
they  didn’t  necessarily  agree. 

Bunch  of  guys  sitting  around  a  white  board  do  the  analysis. 
Break  it  down  into  bite  sized  pieces. 

Warden’s  rings  are  useless.  Going  to  be  a  combination  of 
multiple  models.  Look  at  enemy  and  say,  “this  is  sound.” 

Dream:  live  Barlow  model.  Political  pieces  growing  and 
shrinking. 

Which  model  is  used  is  dependant  upon  the  personality  of  the 
person  using  it. 

Some  models  lend  themselves  to  certain  AORs  better  than 
others. 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re 
going  to  use?  I’ve  never  heard  of  any  of  the  others.  You  look 
at  the  rings  and  you  hand  them  out  to  different  people.  Divide 
them  by  elements  in  the  model  (leadership,  economics,  etc.) 
You  then  decide  what  are  the  COGs  for  the  target  area. 

Warden’s  is  what  they  are  teaching.  But,  we  need  to 
dynamically  react  so  people  can  update  their  buckets. 

1.2 

COG  Visualization  must  support  user- 
guided  SoSA  analysis,  e.g.,  TSA  options 
from  JWAC  in  order  to  determine  COGs 

Who  determines  where  the  COG  is,  say  in  the  case  of 
electrical  power?  -a  complex  target  analysis  should  have  been 
done  already. 

How  is  that  presented  to  the  air  component? 

-Operational  Objectives. 

-JWAC  will  do  analysis  and  present  options. 

We  take  a  snapshot  of  the  DIA  MIDB. 

What  can  we  do  with  this  JWAC  product? 

They  give  you  different  options  for  how  to  take  it  out,  how 
long  it  will  be  down  for,  etc. 
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JWAC  does  SoSA,  not  COG  analysis 

If  I  take  down  the  power,  how  much  does  this  regime  rely  on 
it?  Are  there  other  countries  reliant  on  it? 

There’s  baggage  associated  with  COG.  But  not  target  systems 
analysis.  Yet  they  are  very  similar. 

Strategy  Plans  is  looking  at  the  outcome  of  target  systems 
analysis. 

1.2.1 

COG  Visualization  must  show  direct  and 
indirect  effects  of  COG  within  and 
external  to  the  battlespace  (infers  global 
visualization). 

If  I  take  down  the  power,  how  much  does  this  regime  rely  on 
it?  Are  there  other  countries  reliant  on  it? 

1.3 

COG  Visualization  must  assist  in 
selecting  the  COGs  upon  which  available 
assets  deliver  desired  effects 

System  Req: 

Load  available  asset  capabilities 

Pair  capabilities  to  COG  types 

Visualization  is  implicit 

You’ll  take  your  best  shot  at  understanding  the  enemy 
(PMESII).  Then  you’ll  figure  out  what  the  fastest  way  to  take 
him  down  with  the  assets  I’ve  got.  The  targeteers  will  be 
instrumental  in  determining  this. 

What  should  we  target 

What  can  we  target 

What  do  we  want  to  target? 

Is  on  no  strike  list?  Do  we  have  the  necessary  weapon,  etc.? 

Ultimately  getting  to  weapon  target  pairings.  Think  any  type 
of  target  (person,  place,  thing,  etc.).  Think  any  capability 
(action). 

We  need  to  identify  what  they  rely  on  for  various  things,  like 
power  or  heat. 

2.0 

COG  Visualization  must  allow  for 
identification  of  assumptions  made 
during  identification  of  COGs. 

System  Req: 

ID  assumptions  as  they  are  made  or 
received 

Key  is  to  capture  relevant  data 
without  requiring  repetitive 
effort  from  user 

Track  and  display  assumptions 

If  you  don’t  get  the  guidance,  you  make  it  up  (assumption). 

Known  knowns,  known  unknowns,  and  unknown  unknowns. 

3.0 

COG  Visualization  must  support  the 
current  COG  analysis  processes 

System  Req: 

Support  use  of  common  data  set 

Support  existing  and  user  -defined 
approaches 

Accept  user-defined  approach 
inputs 

Everything  is  being  used,  but  Strange  is  most  common. 
Different  in  every  AOC.  Depends  on  Strategy  Chief 

The  process  is  in  place.  But  there’s  what  should  be,  and 
there’s  what  is. 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re 
going  to  use?  I’ve  never  heard  of  any  of  the  others.  You  look 
at  the  rings  and  you  hand  them  out  to  different  people.  Divide 
them  by  elements  in  the  model  (leadership,  economics,  etc.) 
You  then  decide  what  are  the  COGs  for  the  target  area 

Trving  to  identify  critical  capabilities  and  vulnerabilities,  I  use 
a  svstem  of  svstems  approach. 
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3.1 

COG  Visualization  must  support 
categorization  of  battlespace  situation 
variables  within  any  of  multiple  models 
(e.g.,  bin  each  variable  within  the 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re 
going  to  use?  I’ve  never  heard  of  any  of  the  others.  You  look 
at  the  rings  and  vou  hand  them  out  to  different  people.  Divide 
them  bv  elements  in  the  model  (leadership,  economics,  etc.) 

selected  model) 

System  Req: 

Allow  for  dynamic  situation-dependent 
variable  creation  and  mapping 

You  then  decide  what  are  the  COGs  for  the  target  area. 

Trying  to  identify  critical  capabilities  and  vulnerabilities,  I  use 
a  svstem  of  svstems  approach. 

3.1.1 

COG  Visualization  must  use  models  such 
as  PMESII  to  classify  and  filter 
battlespace  parameters  or  elements  based 
on  model  categories 

Start  with  political.  Constrain  discussion  on  that  until  done. 
Then  we  move  on  to  the  next  one.  Military.  What  are  their 
fielded  forces?  Nighttime  capability?  Currency  of  pilots 

3. 1.1.1 

COG  Visualization  must  support 
breaking  out  PMESII  other  categories 
into  user-definable  subcategories 

The  M  in  PMESII  is  going  to  break  out  into  4  areas: 

Air 

Land 

Sea 

SOF 

3.1.2 

COG  Visualization  must  support  division 
of  labor  within  overall  collaborative 
efforts 

System  Req: 

Support  shared  data 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re 
going  to  use?  I’ve  never  heard  of  anv  of  the  others.  You  look 
at  the  rings  and  vou  hand  them  out  to  different  people.  Divide 
them  by  elements  in  the  model  (leadership,  economics,  etc.) 
You  then  decide  what  are  the  COGs  for  the  target  area 

These  models  are  ways  to  filter,  they  structure  your  work  into 
workable  pieces. 

3.1.3 

COG  Visualization  must  support  tracking 
progress 

COG  Visualization  must  support 
coordinating  work 

Warden’s  model  is  the  most  prevalent.  Is  that  what  we’re 
going  to  use?  I’ve  never  heard  of  any  of  the  others.  You  look 
at  the  rings  and  you  hand  them  out  to  different  people.  Divide 
them  by  elements  in  the  model  (leadership,  economics,  etc.) 
You  then  decide  what  are  the  COGs  for  the  target  area 

3.2 

COG  Visualization  must  allow  for  human 
interaction  in  determination  of  the  COGs. 

Don’t  think  you  can  computer  model  everything.  Some  things 
are  human  centric. 

3.3 

COG  Visualization  must  support 
collaboration  and  federated  sharing  of 
COG  information  and  analysis  data. 

Federated  population  -  not  just  Strategy  Plans. 

I’d  be  happy  if  the  airmen  had  a  seat  at  the  CFACC’s 
planning  table,  (verify  CFC  or  CFACC). 

National  military  strategy,  etc.,  is  it  out  of  our  area?  Yeah. 

Did  we  do  it?  Yeah. 

Don’t  get  wrapped  around  the  crank  about  cross-component 
ripple  effect.  Worry  about  your  own  piece  of  the  mission.  The 
JFC  should  be  worried  about  that. 

3.4 

COG  Visualization  must  assist  in 
capturing  the  process  supported  by 
“whiteboard”  analysis 

Bunch  of  guys  sitting  around  a  white  board  do  the  analysis. 
Break  it  down  into  bite  sized  pieces. 

3.5 

COG  Visualization  must  support 
documentation  of  desired  changes  to 
current  battlespace 

We  then  talk  about  what  each  CO  A  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

106 


3.5.1 

COG  Visualization  must  support  keeping 
the  assessment  and  COA  development 
effort  tied  to  the  desired  end  state 

Are  you  always  focused  on  the  end  state?  Yes. 

3.6 

COG  Visualization  must  support 
development  of  COAs 

Within  COA  development,  we  write  an  operational  or  tactical 
objective  directed  at  the  adversarv,  directed  at  their  COG. 
Alternative  COAs  reallv  depend  on  how  much  we  focus  on 
that  COG. 

3.6.1 

COG  Visualization  must  support 
development  of  operational  and  tactical 
objectives 

Within  COA  development,  we  write  an  operational  or  tactical 
objective  directed  at  the  adversarv,  directed  at  their  COG. 
Alternative  COAs  really  depend  on  how  much  we  focus  on 
that  COG. 

3.6.1. 1 

COG  Visualization  must  support  relating 
these  objectives  to  COGs  and  COAs. 

Within  COA  development,  we  write  an  operational  or  tactical 
objective  directed  at  the  adversarv,  directed  at  their  COG. 
Alternative  COAs  really  depend  on  how  much  we  focus  on 
that  COG. 

3.6.1.2 

COG  Visualization  must  support  and 
document  relating  COGs  to  COAs 

Within  COA  development,  we  write  an  operational  or  tactical 
objective  directed  at  the  adversarv,  directed  at  their  COG. 

Alternative  COAs  really  depend  on  how  much  we  focus  on 
that  COG. 

3.6.2 

COG  Visualization  must  support 
development  of  multiple  alternative 

COAs 

COG  Visualization  must  support  COA 
prioritization 

I  have  alternative  potential  plans,  then  analvze  them  against 
the  enemy,  we  make  a  recommendation.  He  then  chooses  his 
preferred  COA.  They  are  blessed  by  the  commander  and 
weTl  expand  that  and  make  that  the  center  of  the  operational 
plan  for  the  JAOC.  COAs  can  usually  be  preceded  by  the 
word  alternative.  Once  we  choose  one,  that’s  the  plan.  When 
building  one,  we  make  assumptions.  These  assumptions 
become  the  focal  points  of  the  plan.  We  generate  alternatives 
for  each  assumption. 

Within  COA  development,  we  write  an  operational  or  tactical 
objective  directed  at  the  adversary,  directed  at  their  COG. 
Alternative  COAs  reallv  depend  on  how  much  we  focus  on 

that  COG. 

3.6.3 

COG  Visualization  must  support 
selection  of  individual  alternative  COAs 

I  have  alternative  potential  plans,  then  analyze  them  against 
the  enemv,  we  make  a  recommendation.  He  then  chooses  his 
preferred  COA.  Thev  are  blessed  bv  the  commander  and 
we’ll  expand  that  and  make  that  the  center  of  the  operational 

plan  for  the  JAOC.  COAs  can  usuallv  be  preceded  bv  the 
word  alternative.  Once  we  choose  one,  that’s  the  plan.  When 
building  one,  we  make  assumptions.  These  assumptions 
become  the  focal  points  of  the  plan.  We  generate  alternatives 
for  each  assumption. 

3.6.3. 1 

COG  Visualization  must  allow  for  and 
document  approval  of  COA 

I  have  alternative  potential  plans,  then  analyze  them  against 
the  enemv,  we  make  a  recommendation.  He  then  chooses  his 
preferred  COA.  Thev  are  blessed  bv  the  commander  and 
we’ll  expand  that  and  make  that  the  center  of  the  operational 

plan  for  the  JAOC.  COAs  can  usuallv  be  preceded  bv  the 
word  alternative.  Once  we  choose  one,  that’s  the  plan.  When 
building  one,  we  make  assumptions.  These  assumptions 
become  the  focal  points  of  the  plan.  We  generate  alternatives 
for  each  assumption 

3.6.3.2 

COG  Visualization  must  allow 
assumptions  to  be  tied  to  the  alternative 
COAs 

I  have  alternative  potential  plans,  then  analyze  them  against 
the  enemy,  we  make  a  recommendation.  He  then  chooses  his 
preferred  COA.  They  are  blessed  by  the  commander  and 
we’ll  expand  that  and  make  that  the  center  of  the  operational 
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plan  for  the  JAOC.  COAs  can  usually  be  preceded  by  the 
word  alternative.  Once  we  choose  one,  that’s  the  plan.  When 
building  one,  we  make  assumptions.  These  assumptions 
become  the  focal  points  of  the  plan.  We  generate  alternatives 
for  each  assumption 

3.6.4 

COG  Visualization  must  support 
continuous  updating  of  variables  that 
imply  COGs 

System  Req: 

Pub/Sub  (not  pull) 

You  need  to  continually  update  the  things  that  change  that 
might  impact  the  success  of  your  attack  on  those 
vulnerabilities. 

3.6.4.1 

COG  Visualization  must  support 
reanalysis  of  COG  variables  and 
updating  COGs  and  COAs 

System  Req: 

Must  be  machine  focusable  on  impact  to 
success  of  attack  (rule  sets) 

You  need  to  continually  update  the  things  that  change  that 
might  impact  the  success  of  your  attack  on  those 
vulnerabilities. 

3.6.5 

COG  Visualization  must  support 
differentiation  of  COGs  between 
branches  and  sequels  and  between  phases 
of  conflict 

Difference  between  COAs  and  branches  and  sequels. 

Operational  COGs  are  different  by  mission  phase.  That  is 
true  for  both  Red  and  Blue  forces. 

3.6.5. 1 

COG  Visualization  must  support 
selection  of  branch  and  sequel  COAs 

Difference  between  COAs  and  branches  and  sequels 

3.6.6 

COG  Visualization  must  support 
identifying  what  each  COA  is  trying  to 
do 

We  then  talk  about  what  each  COA  is  trving  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

3.6.6.1 

COG  Visualization  must  support 
generation  of  probable  adversary 
response 

We  then  talk  about  what  each  COA  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 

this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

3.6.6.2 

COG  Visualization  must  support 
assessment  of  probable  response  to  COA 

We  then  talk  about  what  each  COA  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

3.6.6.3 

COG  Visualization  must  support 
development/tracking  of  indicators  for 
success  assessment 

We  then  talk  about  what  each  COA  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

3.6.7 

COG  Visualization  must  support 
updating  COAs  based  on  action/reaction 
cycle  (i.e.,  did  the  COA  work  as 
anticipated?). 

We  then  talk  about  what  each  COA  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 

Once  you  identify  your  COGs,  and  you  predict  something 
based  on  an  event  and  they  do  something  different,  you  must 
reanalyze  your  COGs. 

3.6.7.1 

COG  Visualization  must  support 
requesting,  receiving,  and  analyzing 
execution  feedback. 

We  then  talk  about  what  each  COA  is  trying  to  do.  For 
example,  this  COA  will  attack  their  leadership  and  we  think 
this  will  result  in  this  change.  Based  on  that  change,  we  will 
then  do  this  COA. 
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3.7 

COG  Visualization  must  support 
operations  and  operations  other  that  war. 

The  mindset  will  be  different  based  on  mission  type.  I  did 
narcotics  interdiction.  We  looked  for  information  to  give  to 
the  Colombians  so  that  they  can  handle  the  situation.  My 
entire  context  is  different  because  my  operation  is  non-kinetic. 

I  am  in  support  of  SOF  missions. 

3.8 

COG  Visualization  must  support  multiple 
types  of  kinetic  and  non-kinetic 
operations. 

The  mindset  will  be  different  based  on  mission  type.  I  did 
narcotics  interdiction.  We  looked  for  information  to  give  to 
the  Colombians  so  that  they  can  handle  the  situation.  My 
entire  context  is  different  because  my  operation  is  non-kinetic. 

I  am  in  support  of  SOF  missions. 

3.9 

COG  Visualization  must  support  multiple 
types  of  missions. 

The  mindset  will  be  different  based  on  mission  type.  I  did 
narcotics  interdiction.  We  looked  for  information  to  give  to 
the  Colombians  so  that  they  can  handle  the  situation.  My 
entire  context  is  different  because  my  operation  is  non-kinetic. 

I  am  in  support  of  SOF  missions. 

3.10 

COG  Visualization  must  support  the 
different  roles  and  tasks  of  the  different 
COCOMs  (e.g.,  multiple  AORs,  countries 
within  AORs,  AF  tasks  within  AORs, 
etc.) 

I  have  a  problem.  There  is  no  cookie  cutter  solution  for  this. 

It  is  too  situationally  dependent.  Korea  is  different  than 
Columbia  which  is  different  than  India. 

4.0 

COG  Visualization  must  support 
identification  of  restraints  in  attacking  the 
COGs 

Is  on  no  strike  list?  Do  we  have  the  necessary  weapon,  etc.? 

4.1 

COG  Visualization  must  support 
capability  pairing  to  COG 

Is  on  no  strike  list?  Do  we  have  the  necessary  weapon,  etc.? 

5.0 

COG  Visualization  must  support  the 
analyst/planner  in  performing  COG 
analysis  and  decision-making. 

System  Req: 

This  may  be  wizard,  template,  checklist, 
etc. 

By  and  large,  analysts  don’t  have  the  critical  thinking  skills 
needed. 

5.1 

COG  Visualization  must  support  the 
analyst/planner  in  finding  and  correlating 
data 

It  is  typically  ad  hoc  to  figure  out  where  to  get  your  data 

Where  do  you  get  your  info? 

Primary  sources: 

Secret  Service 

State  Department 

You  go  wherever  you  can  to  get  the  information  you  need. 

5.1.1 

COG  Visualization  must  support 
machine-to-machine  ingestion  of  CIA 
World  Fact  Book  formatted  data 

A  lot  of  units  use  the  CIA  World  Fact  Book.  They  break  it 
down  into  the  standard  categories.  That  gives  the  intel 
analysts  a  starting  point  to  go  and  find  out  more  about ...  the 
government,  for  example.  They  might  look  at  literacy  rate  to 
tailor  the  pamphlets  to  that  level. 

5.1. 1.1 

COG  Visualization  must  support 
transforming  data  from  its  original 
categorization  to  the  categorization 
construct  currently  in  use. 

A  lot  of  units  use  the  CIA  World  Fact  Book.  They  break  it 
down  into  the  standard  categories.  That  gives  the  intel 

analysts  a  starting  point  to  go  and  find  out  more  about ...  the 
government,  for  example.  They  might  look  at  literacy  rate  to 
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tailor  the  pamphlets  to  that  level. 

5.1.2 

COG  Visualization  must  support 
ingestion  of  data  from  target  folders 

Another  guv:  I  found  lots  of  information  digging  through 
target  folders 

5.1.2.1 

COG  Visualization  must  interface  to  T- 
Bone/JTT  (System  Req) 

Accessed  SIPRNET  back  to  CENTCOM 

Guvs  at  32”''  AOG  used  those  folders  also. 

5.1.3 

COG  Visualization  must  support  a  CIA 
World  Fact  Book  model 

A  lot  of  units  use  the  CIA  World  Fact  Book.  Thev  break  it 
down  into  the  standard  categories.  That  gives  the  intel 
analysts  a  starting  point  to  go  and  find  out  more  about ...  the 
government,  for  example.  They  might  look  at  literacy  rate  to 
tailor  the  pamphlets  to  that  level. 

5.1.4 

COG  Visualization  must  support 
identification  of  indicators  of  potential 
COGs  and  COAs  within  current  IPB  (see 
templates  and  rule  sets) 

A  lot  of  units  use  the  CIA  World  Fact  Book.  They  break  it 
down  into  the  standard  categories.  That  gives  the  intel 
analysts  a  starting  point  to  go  and  find  out  more  about ...  the 
government,  for  example.  Thev  might  look  at  literacv  rate  to 
tailor  the  pamphlets  to  that  level. 

5.1.5 

COG  Visualization  must  support 
traceability  of  IPB  indicators  to  COG 
determination  and  COA  construction 
(support  for  COA  rationale  and  potential 
for  machine  learning) 

A  lot  of  units  use  the  CIA  World  Fact  Book.  They  break  it 
down  into  the  standard  categories.  That  gives  the  intel 
analvsts  a  starting  point  to  go  and  find  out  more  about ...  the 

government,  for  example.  Thev  might  look  at  literacv  rate  to 
tailor  the  pamphlets  to  that  level. 

5.1.3 

COG  Visualization  must  support  RFI 
generation  and  tracking  and  maintaining 
relationship  to  COGs  and  COAs 

As  an  intel  group,  the  World  Fact  Book  gives  us  our 
generalization.  We  then  use  that  to  go  get  the  nittv  grittv. 

5.1.4 

COG  Visualization  must  support  relating 
RFIs  to  specific  COGs/COAs 

As  an  intel  group,  the  World  Fact  Book  gives  us  our 
generalization.  We  then  use  that  to  go  get  the  nittv  srittv. 

5.1.4.1 

COG  Visualization  must  support 
maintaining  the  relationship  between 
information  requirements  and 
assumptions 

As  an  intel  group,  the  World  Fact  Book  gives  us  our 
generalization.  We  then  use  that  to  go  get  the  nittv  grittv. 

5.1.4.2 

COG  Visualization  must  support 
reachback  to  multiple  sources 

As  an  intel  group,  the  World  Fact  Book  gives  us  our 
generalization.  We  then  use  that  to  go  get  the  nittv  grittv. 

5.1.4.3 

COG  Visualization  must  play  on  the 

GIG  (System  Req) 

As  an  intel  group,  the  World  Fact  Book  gives  us  our 
generalization.  We  then  use  that  to  go  get  the  nittv  grittv. 

5.2 

COG  Visualization  must  support  working 
with  multiple  standardized  user- 
configurable  data  sources 

I  created  mv  own  database  for  the  primarv  targets. 

Included  various  options,  geographic  data,  targeting 
options. 

He  plotted  them  on  maps  using  FalconView  on  ITS. 

5.2.1 

COG  Visualization  must  support  working 
with  multiple  visualization  capabilities 
(e.g.,  geographic,  temporal,  trending, 
managed  knowledge) 

I  created  my  own  database  for  the  primary  targets. 

Included  various  options,  geographic  data,  targeting 

options. 

He  plotted  them  on  maps  using  FalconView  on  ITS. 

5.2.2 

COG  Visualization  must  provide  plug-in 

I  created  my  own  database  for  the  primary  targets. 
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infrastructure  to  support  different  user 
interfaces  (System  Req) 

Included  various  options,  geographic  data,  targeting 
options. 

He  plotted  them  on  mans  using  FalconView  on  ITS. 

5.3 

COG  Visualization  must  support  both  a 
defined  doctrinal  process  (especially  for 
exercises)  and  personal  or  directed 
variants 

In  exercises  thev  try  to  go  bv  doctrine,  which  is  not  what  thev 

do  in  real  life.  In  real  life,  the  personalitv  of  the  leaders 
guides  how  things  are  done. 

5.3.1 

COG  Visualization  must  support  personal 
or  directed  planning  process  variants 

In  exercises  they  try  to  go  by  doctrine,  which  is  not  what  they 
do  in  real  life.  In  real  life,  the  personalitv  of  the  leaders 
guides  how  things  are  done. 

5.4 

COG  Visualization  must  track  the 
transition  from  categorization  of  variables 
(e.g.,  Barlow’s  NEV,  Warden’s  Rings, 
PMESII)  to  assessment  of  variables  (e.g., 
Strange’s  CC-CV-CR,  ENAR)  to  actual 
COA  development 

Trving  to  identify  critical  capabilities  and  vulnerabilities,  I  use 
a  svstem  of  svstems  approach 

5.5 

COG  Visualization  must  support 
identification  and  tracking  of  specific 
information  elements  (situation  variables) 
leading  to  COG  determination  (see 
indicators  req) 

We  need  to  identify  what  they  rely  on  for  various  things,  like 
power  or  heat. 

5.6 

COG  Visualization  must  support 
differentiation  between  elements  that 
analyst/planner  is  responsible  for  tracking 
vs.  elements  for  which  analyst/planner 
must  develop  a  response 

How  do  we  filter  the  data? 

You  are  automatically  limited. 

If  we  are  covering  the  P.  We  are  limited  as  the  Air 
Force.  There’s  the  military  P,  and  within  that  is  what 
the  Air  Force  can  do. 

There  are  parts  of  the  P  problem  that  are  simplv  not 

mv  iob,  but  I  must  pav  attention  to  some  of  those 

other  elements. 

5.6.1. 

COG  Visualization  must  allow  the 
analyst/planner  to  readily  track  elements 
for  which  he/she  is  not  responsible 

How  do  we  filter  the  data? 

You  are  automatically  limited. 

If  we  are  covering  the  P.  We  are  limited  as  the  Air 
Force.  There’s  the  military  P,  and  within  that  is  what 
the  Air  Force  can  do. 

There  are  parts  of  the  P  problem  that  are  simplv  not  mv  iob. 

but  I  must  pav  attention  to  some  of  those  other  elements 

5.6.2 

COG  Visualization  must  allow  the 
analyst/planner  to  update  status  of 
elements  for  which  he/she  is  responsible 

How  do  we  filter  the  data? 

You  are  automatically  limited. 

If  we  are  covering  the  P.  We  are  limited  as  the  Air 
Force.  There’s  the  military  P,  and  within  that  is  what 
the  Air  Force  can  do. 

There  are  parts  of  the  P  problem  that  are  simply  not  my  job, 
but  I  must  pav  attention  to  some  of  those  other  elements 

5.6.3 

COG  Visualization  must  provide 
notification  of  status  changes  whenever 

There  are  parts  of  the  P  problem  that  are  simply  not  my  job, 
but  I  must  pay  attention  to  some  of  those  other  elements 
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the  system  information  updates  itself 

5.7 

COG  Visualization  must  support 
maintaining  a  relationship  between  the 
planner’s  CO  As  and  other  entities’  CO  As 
(emphasis  on  actions). 

Do  you  only  focus  on  the  M? 

No.  We  are  really  only  the  M.  That  is  all  we  have  to 
bear.  Application  of  the  DIME  means  that  we  are 
the  M. 

5.8 

COG  Visualization  must  support  relating 
BLUE  force  capabilities  (assets)  to 
potential  COAs  with  respect  to  the 

COG(s) 

The  matrix  is:  what  do  I  have  [to  bringl  to  bear  in 
theater,  matrixed  with  PMESII.  In  almost  anv  plan, 
you  can  get  to  a  certain  place  in  lots  of  different 
ways. 

5.9 

COG  Visualization  must  portray  both 

RED  and  BLUE  potential  actions 

When  dealing  with  real  world  situations,  vou  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usually  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions.  They  are  trying  to  do  the  same  thing 
to  us.  Within  this,  the  COGs  must  be  identified  that  allow  us 
to  have  the  impact  we  want.  COGs  are  described  as  source  of 
strength  and  the  will  to  use  that  power.  COGs  lie  in  an 
overlap  between  capabilities  and  will. 

If  I’m  trying  to  analyze  a  COG,  he’s  going  to  use  his  actions 
to  oppose  me  (information,  economic,  diplomatic,  military, 
etc.). 

5.9.1 

COG  Visualization  must  support  the 
analyst/planner  relating  COGs  to  a  range 
of  potential  actions 

When  dealing  with  real  world  situations,  you  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usuallv  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions. 

5.9.2 

COG  Visualization  must  support  the 
analyst/planner  relating  proposed  actions 
to  a  range  of  possible  reactions 

When  dealing  with  real  world  situations,  you  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usuallv  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions. 

5.9.3 

COG  Visualization  must  allow  for 
identification  of  ways  to  degrade 
adversary  capabilities 

When  dealing  with  real  world  situations,  you  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usuallv  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions. 

5.9.4 

COG  Visualization  must  allow  for 
identification  of  ways  to  affect  adversary 
will 

When  dealing  with  real  world  situations,  you  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usuallv  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions. 

5.9.5 

COG  Visualization  must  allow  for 
identification  of  ways  to  oppose 
adversary  actions 

When  dealing  with  real  world  situations,  you  must  consider 
this  a  two-sided  problem. 

Our  actions  are  usually  to  degrade  his  capabilities,  affect  his 
will,  oppose  his  actions. 

5.10 

COG  Visualization  must  provide  a  way  to 
display  potential  goals  for  both  RED  and 
BLUE  sides. 

Where  does  PMESII  lie?  This  might  lie  in  the  goals  of  each 
side. 

5.11 

COG  Visualization  must  provide  a  way  to 
map  capabilities  to  adversary  will  and  to 
possible  adversary  actions  dependent 
upon  those  capabilities 

PMESII  conditions  do  not  necessarily  map  to  the  capabilities, 
will,  and  actions  of  the  enemy  in  a  direct  fashion. 

6.0 

COG  Visualization  must  allow 
analyst/planner  to  determine  the  quality 
of  the  IPB. 

If  your  IPB  is  bad,  then  all  subsequent  analysis  is  bad  as  well. 

Known  knowns,  known  unknowns,  and  unknown  unknowns 

6.1 

COG  Visualization  must  support 
documentation  of  perceived  IPB  accuracy 

If  you’re  IPB  is  bad,  then  all  subsequent  analysis  is  bad  as 
well. 
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and  completeness. 

7.0 

COG  Visualization  must  make  IPB 
available  for  COG  analysis. 

If  you’re  IPB  is  bad,  then  all  subsequent  analysis  is  bad  as 
well. 

Primary  vehicle  is  PowerPoint  to  CFACC.  Some  are  250 
slides. 

(on  integration  into  which  tool) 

I  want  a  single  app  that  goes  out  and  gets  all  the  information 
for  me. 

l.l 

COG  Visualization  must  provide  insight 
into  Unknown-Unknowns  (ontological 
references,  templates,  checklists,  and 
other  guidance  tools) 

Known  knowns,  known  unknowns,  and  unknown  unknowns. 

8.0 

COG  Visualization  must  allow 
operational  level  COG  analysis  to 
contribute  to  and  be  correlated  with 
campaign  level  COG  analysis 

We’re  abysmal  at  taking  air  strategy  and  showing  how  it 
contributes  to  the  campaign  strategy. 

Need  to  look  at  supporting  relationships. 

9.0 

COG  Visualization  must  support 
alternative  COG  analysis  conclusions 

CFACC  wants  someone  else  to  have  looked  at  all  this,  even  if 
you’ve  got  a  different  opinion 

Have  we  done  COG  analysis  since  desert  storm?  -yes,  but 
they  didn’t  necessarily  agree. 

10.0 

COG  Visualization  must  support 
escalation  of  required  DIME  actions 
outside  the  air  component  up  the  chain  of 
command 

You’ve  created  a  plan  that  is  heavily  based  on  flying  out  of  a 
certain  air  base.  Then  you’re  told  you  can’t.  Yu  can’t  solve  it 
yourself.  So  it  goes  up  the  chain.  Then  it  might  become  a 
diplomatic  issue. 

11.0 

COG  Visualization  must  support  BLUE 
COG  analysis 

We  have  been  focused  on  the  enemy.  There’s  a  blue  piece  that 
has  been  largely  ignored. 

We’ve  got  the  RED  Barlow  model  (we  want  to  attack).  And 
we’ve  got  the  blue  model  (we  want  to  protect).  We  ignore  the 
blue  model  a  lot. 

12.0 

COG  Visualization  must  leverage 
existing  Operational  Assessment  Team 
products  for  input  to  COG  analysis  and 
status 

Don’t  saddle  the  OA  guys  with  yet  another  deliverable. 

13.0 

COG  Visualization  must  incorporate 
current  visualization  constructs  where 
possible 

Political  vis:  Analyst’s  Notebook  vis.  organizational  chart 

I  created  my  own  database  for  the  primary  targets. 

Included  various  options,  geographic  data,  targeting 
options. 

He  plotted  them  on  maps  using  FalconView  on  ITS. 

13.1 

COG  Visualization  must  support 
geographic,  timeline,  trending,  and  other 
plotting  capabilities 

I  created  my  own  database  for  the  primary  targets. 

Included  various  options,  geographic  data,  targeting 
options. 

He  plotted  them  on  maps  using  FalconView  on  ITS. 

14.0 

COG  Visualization  must  support  all 
levels  of  expertise  with  assistance  such  as 
checklists,  selection  guides,  wizards,  and 
templates. 

What  is  the  education  level?  You  need  a  way  to  document 
these  areas  so  that  if  they  become  important,  we  see  them.  A 
checklist  stvle  format  might  work. 

A  drop  down  menu  of  Warden,  Strange,  Barlow,  PMESII.  If 
you  give  the  operator  this  option,  will  they  know  which  to 
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use?  We  can  provide  guidance  regarding  which  to  use. 

For  me  to  accomplish  mv  mission,  I  need  to  identifv  mv  end 
goal  and  work  backwards.  Example,  food  following  a 
tsunami.  How  do  I  get  enough  food?  How  do  I  protect  these 
people?  How  do  I  protect  them  from  neighbors?  What  do 
their  neighbors  have  that  could  threaten  them?  What  might  I 
do  about  those? 

14.1 

COG  Visualization  must  allow  for 
identification  of  desired  end  states  and 
support  working  backwards  to 
requirements  to  reach  the  desired  end 
states. 

For  me  to  accomplish  mv  mission,  I  need  to  identifv  mv  end 
goal  and  work  backwards.  Example,  food  following  a 
tsunami.  How  do  I  get  enough  food?  How  do  I  protect  these 
people?  How  do  I  protect  them  from  neighbors?  What  do 
their  neighbors  have  that  could  threaten  them?  What  might  I 
do  about  those? 

15.0 

COG  Visualization  must  help  move  COG 
analysis  from  subjective  to  objective 

*PMESII  mapping  is  all  subjective 

Look  at  each  element,  what  do  we  know,  capability, 
many  details 

*Huge  human  element,  so  need  to  less  subjectively 
and  more  objectively  define  the  problem 

What  are  potential  consequences  of  PMESII 
situations 

Map  PMESII  information  to 

OO/COGs/effects  to  aid  analyst  critical 
thinking 

Viz  will  not  be  main  focus  (maybe  wizard- 
based  approach,  e.g..  Turbo  Tax) 

15.1 

COG  Visualization  must  allow  for 
identification  of  subjective  vs.  objective 
assessments 

*PMESII  mapping  is  all  subjective 

Look  at  each  element,  what  do  we  know,  capability, 
many  details 

*Huge  human  element,  so  need  to  less  subjectivelv 
and  more  objectivelv  define  the  problem 

15.2 

COG  Visualization  must  allow  for 
identification  of  potential  consequences 
of  PMESII  situations 

What  are  potential  consequences  of  PMESII 
situations 

Map  PMESII  information  to 

OO/COGs/effects  to  aid  analvst  critical 
thinking 

Viz  will  not  be  main  focus  (mavbe  wizard-based  approach, 
e.g..  Turbo  Tax) 

15.3 

COG  Visualization  must  map  PMESII 
information  to  OOs/COGs/effects 

What  are  potential  consequences  of  PMESII 
situations 

Map  PMESII  information  to 

OO/COGs/effects  to  aid  analvst  critical 
thinking 

Viz  will  not  be  main  focus  (maybe  wizard-based  approach, 
e.g..  Turbo  Tax) 

15.4 

COG  Visualization  must  provide 
guidance  on  modeling  and  analysis 
techniques 

What  are  potential  consequences  of  PMESII 
situations 

Map  PMESII  information  to 
OO/COGs/effects  to  aid  analyst  critical 
thinking 

Viz  will  not  be  main  focus  (mavbe  wizard-based  approach, 
e.g..  Turbo  Tax) 

16.0 

COG  Visualization  must  support  nodal 
analysis 

How  do  I  show  you  all  of  this?  Is  it  strict  nodal  analysis? 
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Show  nodes  with  most  things  going  to  it? 

Show  nodes  with  the  most  critical  connections? 

16.1 

COG  Visualization  must  provide  multiple 
algorithms  for  nodal  analysis  to  provide 
flexibility  in  views  and  assessments 

I  could  have  something  really  important  that  has  only  one 
relationship.  That  could  be  the  most  important  node  and  it  has 
only  one  pathway. 

16.2 

COG  Visualization  must  allow  flexibility 
in  the  analytical  process  and  level  of 
detail  required  to  complete  the  planning 
task 

I  can  imagine  myself  putting  these  pieces  together,  but  whaf  s 
the  point?  We  need  to  maintain  focus  on  the  end  state.  If  the 
answer  bubbles  to  the  top,  whv  waste  time  filling  out  a 

model? 

16.2.1 

COG  Visualization  must  flag  rapid 
decisions  made  during  a  compressed 
planning  cycle 

I  can  imagine  myself  putting  these  pieces  together,  but  whaf  s 
the  point?  We  need  to  maintain  focus  on  the  end  state.  If  the 
answer  bubbles  to  the  top,  whv  waste  time  filling  out  a 

model? 

16.2.2 

COG  Visualization  must  allow  for 
assessment  of  and  feedback  on  rapid 
decisions  made  during  a  compressed 
planning  cycle 

I  can  imagine  myself  putting  these  pieces  together,  but  whaf  s 
the  point?  We  need  to  maintain  focus  on  the  end  state.  If  the 
answer  bubbles  to  the  top,  whv  waste  time  filling  out  a 

model? 

16.2.3 

COG  Visualization  must  allow  for  rapid 
analysis  of  COGs  that  bubble  to  the  top 

I  can  imagine  myself  putting  these  pieces  together,  but  whaf  s 
the  point?  We  need  to  maintain  focus  on  the  end  state.  If  the 
answer  bubbles  to  the  top,  whv  waste  time  filling  out  a 

model? 

16.2.4 

COG  Visualization  must  allow  for 
identification  of  levels  of  confidence  in 
solutions 

I  can  imagine  myself  putting  these  pieces  together,  but  whaf  s 
the  point?  We  need  to  maintain  focus  on  the  end  state.  If  the 
answer  bubbles  to  the  top,  whv  waste  time  filling  out  a 

model? 

16.3 

COG  Visualization  must  provide  methods 
to  visualize  and  denote  lines  of  effect 

I  like  the  Red  Thompson’s  lines  of  effect.  That  works  for  me. 

It  provides  a  roadmap  for  where  you  might  go 

17.0 

COG  Visualization  must  support 
“brainstorm-style”  collaborative  work 
with  shared  visualizations  and  real-time 
communication 

“I  will  never  get  on  a  computer  to  get  this  going.  I  grab 
creative  people,  get  together,  and  brainstorm.  To  be 
constrained  by  a  computer  program  or  tool  is  not  conducive  to 
what  I  try  to  do  at  the  beginning.” 

18.0 

COG  Visualization  must  pull  information 
from  “lessons  learned”  (e.g.,  mission 
analyses,  analyst  notes,  etc.)  and  add  to 
existing  knowledge  base  to  update 
analyst  guidance 

Strategy  Plans  must  revisit:  Known-Knowns,  Known- 
Unknowns,  Unknown-Unknowns 

VPT-like  system  infers  the  unknown  unknowns  and 
flags  analyst  attention 

19.0 

COG  Visualization  must  accommodate 
uncertainty,  incomplete  analyses,  and 
fuzzy  concepts 

COG  has  to  accommodate  squishy  as  well  as  firm 
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Introduction 

This  report  documents  a  process  utilized  to  generate  initial  interface  design  concepts  for 
visualization  of  the  air  tasking  order  (ATO)  within  the  combat  planning  cell.  This  project  began  as 
a  two-pronged  effort:  data  wall  concepts  and  the  development  of  visualizations  for  use  within  the 
Strategy  Plans  cell.  As  the  data  analysis  progressed,  the  focus  moved  to  the  ATO  visualization, 
only  with  emphasis  starting  at  the  Strategy  Plans  cell  and  settling  on  the  combat  planning  cell. 
Though  those  shifts  in  focus  were  costly  in  terms  of  time  and  budget,  they  were  a  valuable  process 
to  go  through  for  both  the  Government  and  SRA  International,  Inc.  (SRA).  This  discovery  process 
allowed  us  to  identify  an  area  in  critical  need  of  visualization  and  support  tools.  That  area,  the 
combat  planning  cell  within  the  AOC,  is  dramatically  under  supported  in  terms  of  tools  that  support 
cognition,  decision  making,  workload,  and  data  input.  The  combat  operations  cell  tends  to  receive 
the  most  attention,  while  the  planning  cell  is  often  ignored.  Although  it’s  not  the  focus  of  this 
effort,  future  tool  development  must  focus  on  the  collaborative  elements  that  exist  between  combat 
operations  and  planning.  We  understand  tools  are  currently  being  developed,  we  can  only  hope  that 
these  tools  contain  features  that  support  collaborative  team  performance  as  well  as  individual  task 
completion. 
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Cognitive  Task  Anaiysis 

Cognitive  Task  Analysis  (CTA)  is  a  set  of  methods  aimed  at  describing  the  cognition  required 
for  task  performance.  This  report  describes  a  CTA  process  used  to  understand  the  cognitive 
demands  of  personnel  operating  within  a  planning  cell  of  the  Air  Operations  Center  (AOC).  The 
first  round  of  interviews  was  conducted  with  members  of  the  Strategy  Plans  cell.  Following  that 
data  collection  effort,  the  team  chose  to  focus  on  the  Combat  Plans  cell.  Following  this  decision, 
we  interviewed  individuals  from  that  cell  in  an  attempt  to  capture  their  decisions,  judgments,  and 
other  cognitive  demands  associated  with  optimal  performance  within  Combat  Plans. 

In  a  general  sense,  CTA  provides  methods  for  the  researcher  to  identify  the  cognitive 
processes  that  underlie  skilled  performance  of  tasks.  These  cognitive  processes  can  include  the 
ability  to: 


•  control  attention, 

•  use  working  memory  and  long-term  memory, 

•  make  perceptual  discriminations  between  subtle  cues  and  patterns, 

•  form  situation  awareness, 

•  construct  mental  models, 

•  apply  strategies  and  heuristics  to  make  decisions,  solve  problems,  and  plan, 

•  derive  inferences, 

•  recognize  typical  and  anomalous  events, 

•  monitor  and  adapt  cognitive  processes  (metacognition). 

CTA  techniques  and  methods  are  designed  for  capturing  each  of  these  cognitive  processes. 
The  specific  CTA  techniques  employed  depend  on  the  goals  of  the  study,  the  domain  being  studied, 
and  the  cognitive  processes  of  greatest  interest  to  the  researcher.  For  this  project,  SRA  employed 
CTA  techniques  to  focus  on  the  decisions  made  by  individuals  in  the  AOC. 

This  project  applied  this  process  to  generate  a  set  of  decision  requirements  tables  and  concepts 
to  aid  combat  planners  to  visualize  and  develop  an  air  tasking  order  (ATO).  Due  to  scheduling 
and/or  access  issues,  we  were  only  able  to  conduct  a  small  set  of  interviews.  Yet,  these  interviews 
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provided  great  insight  into  the  process  and  cognitive  requirements  necessary  for  successful 
completion  of  the  tasks.  We  were  also  fortunate  to  attend  a  set  of  focus  groups  in  which  we  were 
able  to  generate  discussion  regarding  the  current  processes,  as  well  as  to  gain  feedback  on  “how  it 
should  be.”  During  these  sessions,  we  asked  the  focus  groups  to  imagine  a  perfect  tool  suite  to  help 
with  planning.  Their  brainstorming  sessions  provided  innovative  ideas  that  influenced  our 
visualization  concepts. 

This  limited  set  of  interviews  required  a  reliance  on  several  publications  in  order  to  support  the 
CTA.  These  readings  provided  an  understanding  of  the  desired  process  for  ATO  planning  as  well  as 
doctrinal  information  regarding  the  interactions  of  combat  plans  with  other  AOC  members  (see 
Appendix  A).  However,  these  readings  did  not  provide  enough  decision  context.  They  simply 
provided  background  information.  The  interviews  were  needed  to  bring  the  cell  to  life,  to  learn 
about  the  decision  process,  and  to  understand  where  things  work  well  and  where  they  break  down. 

The  data  analysis  was  guided  by  several  models,  including  the  Recognition-Primed  Decision 
Model  (RPD)  (Klein,  1989),  the  Advanced  Team  Decision  Making  (ATDM)  model  (Zsambok, 
Klein,  Kyne,  &  Klinger,  1993),  and  several  theories  on  problem  solving  (Bennett,  &  Flach,  1992 
and  Jones  &  Mitchell,  1995).  These  models  and  theories  provided  the  necessary  focus  for  the  team 
to  rapidly  develop  the  data  that  appears  in  the  Appendices  of  this  document. 

The  RPD  model  was  developed  based  on  field  studies  of  the  way  experienced  personnel 
actually  make  decisions.  The  model  explains  how  people  can  use  experience  to  react  rapidly,  and 
how  they  can  make  good  decisions  without  having  to  contrast  options.  The  model  has  been  tested 
and  has  been  supported  by  different  research  teams  working  in  a  great  variety  of  settings. 

The  RPD  model  attempts  to  describe  what  people  actually  do  under  conditions  of  time 
pressure,  ambiguous  information,  ill-defined  goals,  and  changing  conditions.  The  model  focuses  on 
experienced  agents,  working  in  complex,  uncertain  conditions,  who  face  high  consequences  for 
their  actions.  The  model  addresses  situation  awareness  and  problem  solving  as  a  part  of  the 
decision-making  process. 
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The  Advanced  Team  Decision  Making  (ATDM)  model  includes  ten  behaviors  critical  to  a 
team’s  success.  Developed  within  a  project  for  the  Army,  the  model  is  applicable  for  strategic  and 
tactical  teams,  as  well  as  planning  and  action  teams.  The  model  has  been  evaluated  at  several 
locations  within  the  military  (i.e..  Army  War  College  and  Industrial  College  of  the  Armed  Forces) 
and  has  been  applied  to  wide  variety  of  domains  (Klinger  &  Klein,  1999).  The  ATDM  model  is 
organized  around  three  critical  team  elements: 

•  Team  Identity 

•  T earn  Self  Monitoring 

•  Team  Conceptual  Level 

The  ATDM  model  is  not  an  all-inclusive  model  of  how  teams  perform.  Instead,  it  provides  guidance 
as  to  the  behaviors  that  matter  most.  Successful  teams  exhibit  the  10  behaviors  identified  in  the 
model,  and  less  successful  teams  do  not  exhibit  one  or  more  of  those  behaviors.  This  model, 
therefore,  is  a  valuable  tool  for  assessing  team  performance.  Using  it  as  a  framework,  one  can 
bypass  unnecessary  team  process  issues  and  focus  on  those  that  matter  most. 

The  two  elicitation  methods  used  during  this  effort  are  described  in  the  following  sections. 
Those  methods,  the  Team  Knowledge  Audit  (TKA)  (Klinger,  2003)  and  the  Critical  Decision 
Method  (CDM)  (Klein,  Calderwood,  &  MacGregor,  1989),  were  selected  based  on  the  data  these 
methods  provide. 

Specifically,  the  TKA  employs  a  set  of  probes  designed  to  describe  types  of  domain 
knowledge  or  skill  and  elicit  appropriate  examples.  The  goal  is  not  simply  to  find  out  whether  each 
component  is  present  in  the  task,  but  to  find  out  the  nature  of  the  skills,  specific  events  where  they 
were  required,  strategies  that  have  been  used,  and  so  forth.  The  probes,  which  stem  from  the 
behaviors  described  in  the  ATDM  model,  are  the  starting  points  for  conducting  this  interview.  From 
the  probes,  real-world  examples  can  be  elicited.  Then,  the  interviewer  asks  for  specifics  about  the 
examples  in  terms  of  critical  cues  and  strategies  of  decision  making.  This  is  followed  by  a 
discussion  of  potential  errors  that  a  novice,  less-experienced  team  member  might  have  made  in  this 
situation.  The  examples  elicited  with  the  Team  Knowledge  Audit  do  not  contain  the  extensive  detail 
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and  sense  of  dynamies  that  more  labor-intensive  methods  sueh  as  the  Critical  Decision  Method 
incident  accounts  often  do.  However,  they  do  provide  enough  detail  to  retain  the  appropriate  context 
of  the  incident. 


The  CDM  technique  provides  much-needed  context  to  the  decision  process.  A  CDM  session 
is  organized  around  a  specific  event  in  which  the  team  member  has  played  an  active  role.  The 
interviewers  elicit  an  account  of  the  incident,  and  then  drill  progressively  deeper  into  several  aspects 
of  the  event  in  order  to  uncover  the  decisions  and  judgments  made  during  the  incident;  the  cues, 
factors,  and  other  types  of  information  employed;  and  the  interaction  that  occurred  with  other  team 
members.  The  goal  of  the  CDM  is  to  elicit  a  context-rich  example  of  how  the  team  member 
functions  on  the  team.  The  value  of  the  CDM  lies  in  collecting  a  set  of  highly  detailed  incident 
accounts  from  various  individuals  with  varying  levels  of  expertise.  Taken  together,  CDM  data 
provide  an  excellent  description  of  the  cognition  involved  in  the  completion  of  tasks  and  subtasks, 
as  well  as  the  information-sharing  and  decision-making  required  for  team  task  performance. 

The  Cognitive  Task  Analysis  consisted  of  several  phases: 

•  Focus  groups  were  used  at  the  pre- Warfighter  Analysis  Workshop  (WAW) 

o  Time  was  split  between  discussions  regarding  the  application  of  a  data  wall  and 
conceptual  discussions  regarding  ATO  visualization  display  designs 

•  Nine  interviews  were  conducted  at  505*  TRS,  Hurlburt,  AFB 

o  A  mixture  of  Team  Knowledge  Audit  and  Critical  Decision  Method  was  enlisted 
o  Interviews  were  also  split  between  information  regarding  the  application  of  a  data 
wall  and  potential  methods  for  application  within  ATO  visualization  displays 
o  Raw  notes  from  these  interviews  are  included  in  Appendix  B. 

•  Two  interviews  were  held  with  Subject  Matter  Experts  (SMEs)  at  SRA 

o  Informal  interviews  and  brainstorming  sessions  were  also  employed 
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Summary  of  Findings 

Combat  Planners  are  tasked  with  refining  high  level  planning  guidance  passed  to  them  from 
the  Strategic  Planning  cell.  In  Combat  Plans,  individuals  develop  fairly  detailed  plans  regarding 
packages,  missions,  and  targets.  At  the  outset,  we  focused  on  usability  issues,  as  well  as  helping 
them  with  resource  allocation  in  the  areas  of  tankers  and  suppression  of  enemy  air  defense  (SEAD) 
assets.  Through  the  course  of  the  effort,  we  began  to  realize  that  individuals  start  with  targets  and 
work  “out.”  That  is,  they  organize  their  work  around  targets  and  move  out  from  there  in  terms  of 
aircraft  allocation  (both  fighters  and  support).  Given  this  finding,  we  modified  our  visualization 
concepts  to  better  support  their  process. 

Our  analysis  of  the  data  provided  us  with  the  following  high  level  areas  of  focus  for  a 
visualization  tool. 

•  Targeting  Decisions 

o  Munitions 
o  Packages 
o  Timing 
o  Coordination 

•  Allocation  Decisions 

o  Tankers 
o  SEAD 
o  Units 

•  Visualization  issues 

o  Fly/No  fly  zones 
o  Weather 
o  Enemy  locations 

•  Usability  Issues 

o  Minimize  typing  (“fat  fingering”) 

■  More  drag  and  drop 

o  Better  utilization  of  mouse  clicks 

■  Right  click  for  more  information 


122 


o  Better  use  of  “mouse  overs” 
o  Ability  to  save  preferences 

o  Ability  to  quickly  access  relevant  portions  of  previous  ATOs 
■  Ability  to  cut  and  paste  from  previous  ATOs 

Appendix  C  provides  initial  visualization  concepts  and  supporting  process  descriptions  for  Combat 
Plans. 
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Conclusions 

Combat  Planners  are  faced  with  classical  naturalistic  decision  making  issues.  They  operate  in  an 
environment  of  high  time  pressures,  high  stakes,  limited  resources,  conflicting  goals,  uncertainty, 
and  organizational  constraints.  The  systems  they  use  require  massive  user  input  and  dramatically 
increase  workload,  chance  for  error,  and  distraction.  These  systems  do  little  to  support  critical 
cognitive  elements  that  are  intrinsic  to  plarming.  Current  systems  don’t  let  planners  ask  “what  if?” 
questions,  i.e.  they  don’t  support  replarming,  efficient  resource  allocation,  nor  do  they  provide 
adequate  visual  representations,  including  simulating  the  entire  mission  in  a  time  compressed 
manner.  Future  systems  must  consider  these  critical  cognitive  elements  at  the  outset  of 
development. 

At  the  collaborative  level,  much  work  needs  to  be  done  to  identify  the  critical  collaborative 
elements  across  all  planning  cells,  combat  cells,  and  assessment  cells.  Simply  generating  one 
system  for  all  will  not  solve  the  problem.  For  example,  how  much  of  the  “behind  the  scenes” 
contingency  development  should  be  passed  with  the  plan  into  operations?  What  is  the  best  method 
for  transferring  information  from  the  strategic  planners  to  combat  planners?  What  data  should  be 
transferred?  Should  individuals  involved  with  the  plan  move  with  that  plan  as  it  goes  from  cell  to 
cell?  How,  and  what  type,  of  feedback  or  information  should  come  back  to  combat  and  strategic 
planning  during  and  after  mission  execution?  How  should  that  information  returning  to  combat  and 
strategy  plans  be  catalogued,  used,  etc? 

This  effort  has  provided  a  starting  point  for  the  development  of  effective,  performance  changing 
visualization  tools  for  use  within  combat  plans.  At  a  time  when  reduced  manpower  and  increases  in 
technology  are  at  the  center  of  nearly  all  research  efforts,  a  user-focused  process  has  begun  to 
minimize  the  impact  of  manpower  reduction  while  still  increasing  performance.  The  future  of  tool 
development  will  require  a  more  extensive  analysis  of  the  tasks,  cognitive  elements,  and  workload 
issues  within  all  planning  cells.  From  there,  collaborative,  cognitively  aligned  tools  can  be 
developed  that  could  radically  increase  operator  and  team  performance. 
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Appendix  A.  Combat  Plans  and  Combat  Operations  Division 
Understanding 

Combat  Plans  Division  Personnel  Description 

Chief  of  Combat  Plans  (CCP) 

Targeting  Effects  Team  (TET)  Tasks 

1.  Task-organize  personnel  assigned  or  attaehed  for  augmentation,  to  optimize  the  specific 
contributions  of  individual  officers,  civilians,  and  duty  technicians. 

2.  Ensure  personnel  receive  sufficient  "spin  up"  training  to  accomplish  the  mission. 

3.  Establish  procedures  to  ensure  the  TET  provides  complete,  accurate,  properly  formatted,  and 
timely  inputs  to  theater  battle  management  applications  using  standard  formats.  Effect 
quality  control  measures  to  ensure  accuracy  of  data  inputs,  worksheets,  and  other  baseline 
planning  materials. 

4.  File  and  store  critical  JIPTL  planning  materials  and  all  published  documents,  to  include 
detailed  target  worksheets,  briefings,  and  any  supporting  working  papers. 

5.  Maintain  access  to  joint  target  list  (JTL),  RTL,  and  NSL. 

6.  Develop  the  C/JFC's  JIPTL.  Obtain  the  integrated  TNL  from  the  Target  Development  Cell 
as  the  basis  for  formulating  the  daily  targeting  plan. 

7.  10  planning  will  be  based  on  overall  C/JFC  and  COMAFFOR  or  C/JFACC  objectives. 
Creative  cross-flow  of  information  can  enable  dynamic  conventional  and  non-conventional 
planning  scenarios.  10  actions,  particularly  those  directly  supporting  a  specific  ATO 
mission,  require  a  high  level  of  integration  into  the  ATO  timeline.  10  integration  with  all 
other  operations  across  the  CPD  and  COD  ensures  synergistic  effects  to  most  effectively 
cripple  the  enemy's  war  fighting  capability. 

8.  Synchronize  air  and  space  targeting  among  the  respective  components.  Provide  a  macro¬ 
level  feasibility  review,  coordinate,  and  deconflict  initial  mission  planning  across  the 
components.  Identify  and  assign  targets  uniquely  suited  for  attack  by  air  and  space 
resources  that  need  to  be  tasked  prior  to  the  MAAP,  as  well  as  targets  best  suited  for 
supporting  component  efforts  in  the  deep  battle. 

9.  Validate  targeting  solutions  for  kinetic  and  non-kinetic  attack  to  achieve  desired  effects 
against  prioritized  targets. 

10.  Validate  weaponeering  solutions  for  achieving  desired  effects  against  selected 
DMPIs/targets. 

11.  Coordinate  with  the  C2  Plans  Team  Chief  to  ensure  the  MAAP  Team  uses  the  appropriate 
ACO,  JCEOI,  and  JRFL. 

12.  Review  all  previous  assessments  pertaining  to  current  JIPTL  development. 

The  TET  reports  to  the  CCP. 

Members  are: 

•  TET  Chief 

•  Target  Planners 

•  ATO  Coordinators 

•  Information  Operations  Planner 
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•  JAG 

•  Space  Planner 

•  EW  Planner 

•  Collection  Manager 

Master  Air  Attack  Plan  (MAAP)  Team  Tasks 

1.  Task-organize  personnel  assigned  or  attaehed  for  augmentation,  to  optimize  the  specific 
contributions  of  individual  officers,  civilians,  and  duty  technicians. 

2.  Ensure  personnel  receive  sufficient  "spin  up"  training  to  accomplish  the  mission. 

3.  Establish  the  MAAP  Team  battle  rhythm  for  sustained  execution  planning. 

4.  Establish  procedures  to  ensure  the  MAAP  Team  review  the  most  current  version  of  the 
ROW,  all  detailed  execution  plans,  and  supporting  SPINS  required  to  develop  the  daily 
MAAP. 

5.  Ensure  the  MAAP  Team  develops  relevant  SPINS  as  required. 

6.  Develop  MAAP  Team  processes  to  facilitate  timely  generation  of  the  daily  MAAP. 

7.  Establish  procedures  to  ensure  the  MAAP  Team  provides  complete,  accurate,  properly 
formatted,  and  timely  inputs  to  theater  battle  management  applications  using  standard  ATO 
format.  Effect  quality  control  procedures  to  ensure  accuracy  of  data  inputs,  worksheets,  and 
other  baseline  planning  materials. 

8.  Coordinate  STO  capabilities  into  combat  planning. 

9.  Coordinate  with  ATO  Production  Team  Chief  to  attain  a  working  ABP. 

10.  Coordinate  with  the  C2  Plans  Team  Chief  to  ensure  the  MAAP  Team  uses  the  most  current 
ACO,  JCEOI,  and  JRFL. 

1 1 .  Provide  the  TET  with  air  and  space  resource  capabilities  for  estimating  DMPI  servicing 
capability. 

12.  Ensure  support  from  METOC  and  space  personnel  in  the  development  of  the  MAAP. 

13.  Develop  a  standard  MAAP  brief  to  obtain  C/JFACC  approval  prior  to  ATO  production. 

14.  Review  the  most  current  version  of  the  ROW,  all  detailed  execution  plans,  and  supporting 
SPINS  required  to  develop  the  daily  MAAP. 

15.  Develop  and  update  a  sortie  flow  plan  for  all  C/JFACC  assets  based  on  sustainable  aircraft 
generation  rates  and  utilization  during  a  nominal  ATO  period. 

16.  Achieve  situational  awareness  to  enable  effective  planning  by  reviewing  all  relevant 
information  regarding  the  mission,  battlespace,  and  resources. 

17.  Develop  the  overarching  MAAP  for  the  particular  ATO  period  considering  the  effects-based 
requirements,  operational  context,  and  environment.  Define  initial  force  employment 
packages  to  include  support  requirements,  approximate  target  areas  and  vulnerability 
windows,  sequence  of  attacks,  and  flow  of  air  and  space  forces  into  and  from  the  target 
areas. 

18.  Plan,  coordinate,  and  task  assets  available  for  C/JFACC  tasking,  as  required. 

The  MAAP  team  reports  to  the  CCP. 

Members  are: 

•  MAAP  Team  Chief 

•  MAAP  Planners 

Air  Tasking  Order  (ATO)  Production  Team  Tasks 

1.  Task-organize  ATO  Production  Team  personnel  assigned  or  attached  for  augmentation,  to 
optimize  the  specific  contributions  of  individual  ATO  duty  officers  and  duty  technicians. 
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2.  Ensure  ATO  Production  Team  personnel  receive  sufficient  "spin-up"  training  to  accomplish 
the  mission. 

3.  Establish  the  ATO  Production  Team  battle  rhythm  for  sustained  ATO  production. 

4.  Establish  procedures  to  ensure  ATO  Production  Team  personnel  review  the  most  current 
version  of  the  ROE,  all  detailed  execution  plans,  and  supporting  SPINS,  as  required,  to 
develop  and  produce  the  ATO/ACO. 

5.  Ensure  the  ATO  Production  Team  creates  and  maintains  accurate  planning  databases  in  the 
theater  battle  management  system  and/or  applications. 

6.  Ensure  the  ATO  Production  Team  supports  compilation  and  publication  of  relevant  SPONS, 
as  required. 

7.  Develop  ATO  Production  Team  processes  to  facilitate  timely  production  of  the  daily  ATO. 

8.  Develop  effective  quality  control  procedures  and  conduct  a  comprehensive  ATO  review 
prior  to  obtaining  C/JFACC  approval  and  ATO  publication  and  dissemination. 

9.  Establish  procedures  to  track  transmission  and  timely  receipt  of  the  ATO. 

10.  Import  SPINS,  to  include  any  created  in  the  RSTA  Annex,  as  required. 

11.  Coordinate  and  input  to  ATO  air  refueling  assets  available  for  C/JFACC  tasking. 

12.  Develop  and  maintain  a  comprehensive  address  list  of  approved  ATO  recipients  and 
coordinate  redundant  procedures  for  ATO  dissemination  with  the  communication  support 
team. 

13.  After  obtaining  approval  for  release,  disseminate  the  ATO  to  tasked  units  and  agencies  by 
the  most  expeditious  means  available. 

14.  File  and  store  critical  planning  materials  and  all  published  documents  to  include  the  daily 
ABP,  ATO,  ACO,  as  well  as  the  supporting  SPINS. 

The  ATO  production  team  reports  to  the  CCP. 

Members  are: 

•  ATO  Production  Team  Chief 

•  Non-commissioned  Officer  in  Charge  (NCOIC)  ATO  Production  Team 

•  TBMCS  AODB  Manager 

•  ATO  Production  Team  Technicians 

C2  Planning  Team  Tasks 

1.  Task-organize  C2  Planning  Team  personnel  assigned  or  attached  for  augmentation,  to 
optimize  the  specific  contributions  of  individual  duty  officers  and  duty  technicians. 

2.  Ensure  C2  Planning  Team  personnel  receive  sufficient  "spin  up"  training  to  accomplish  the 
mission. 

3.  Establish  the  C2  Planning  Team  battle  rhythm  for  sustained  ACO  production. 

4.  Establish  procedures  to  ensure  C2  Planning  Team  personnel  review  the  most  current  version 
of  the  ROE,  all  detailed  execution  plans,  and  supporting  ATO  SPINS  required  to  develop 
detailed  execution  plans  and  produce  the  ACO. 

5.  Ensure  the  C2  Planning  Team  creates  and  maintains  an  accurate  planning  database  of  the 
ACMs  in  the  theater  battle  management  application. 

6.  Ensure  the  C2  Planning  Team  develops  C2  SPINS  as  required,  incorporating  host  nation, 
allied,  and  other  Service  inputs  into  appropriate  portions  of  the  ATO,  SPINS,  and  data  link 
tasking  documentation. 

7.  Develop  C2  Planning  Team  processes  and  establish  procedures  to  collect  inputs  and  develop 
detailed  execution  plans,  to  include  the  ACP  and  ADP. 
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8.  Establish  procedures  to  evaluate  existing  airspace  control  systems  and  determine  changes 
necessary  to  fulfill  air  traffic  control  requirements,  ensuring  seamless  integration  with  air 
defense,  theater  missile  defense  (TMD),  and  joint/combined  air  and  space  operations. 

9.  Ensure  the  C2  Planning  Team  inputs  are  complete,  accurate,  properly  formatted,  and  timely 
ACMs  to  theater  battle  management  applications  using  standard  AGO  formats. 

10.  Airspace  Management. 

11.  Air  Defense  Planning.  The  C2  Planning  Team  prepares  comprehensive  air  and  missile 
defense  plans.  Based  on  the  availability  of  airborne  and  ground-based  weapon  systems,  data 
link  architectures,  and  tactical  C2  relationships,  the  primary  outputs  of  this  effort  are  the 
ADP,  TACOPDAT  and  OPTASK  LINK  messages. 

12.  C2  Architecture  Planning.  The  C2  architecture  planners  develop  the  C2  architecture  to 
support  air  and  space  operations. 

13.  C2  Communications  Planning.  The  C2  communication  planners  work  closely  with  all  joint 
flying,  C2,  and  airspace  management  elements  tasked  in  ATO/ACO  to  collect  their 
communication/frequency  requirements.  They  in  turn  coordinate  these  requirements  with 
the  C/JFACC  and  AOC  frequency  manager  for  inclusion  in  the  overall  JCEOI.  The  C2 
communication  planners  will  coordinate  with  the  frequency  manager  for  all  necessary 
frequencies  and  call  signs  to  build  the  supporting  communications  sections  of  the  SPINS 
portion  of  the  ATO  and  to  provide  to  the  ATO  mission  planners. 

The  C2  Planning  Team  reports  to  the  CCP. 

Members  are: 

•  C2  Planning  Team  Chief 

•  C2  Air  Defense 

•  C2  Architecture 

•  Data  Link  Architecture 

•  Airspace  Managers 

•  C2  Communications  Planners 

•  Air  Support  Planners 

•  Augmented  by  other  Service  and  Component  liaisons 

Combat  Operations  Division  Personnel  Description 

Chief  of  Combat  Operations  (CCO) 

Offensive  Operations  Team  Tasks 

1.  Supervise  offensive  operations  during  eaeh  shift  with  special  emphasis  on  integrating  all 
offensive  and  support  operations. 

2.  Monitor  the  current  offensive  air  and  space  operations  and  advise  the  CCO  or  SODO  of 
dynamie  mission  requirements  and  resouree  status. 

3.  Reeommend  immediate  ehanges  to  the  ATO  when  the  situation  dietates. 

4.  Assist  CCO  and  SODO  in  pre-employment,  execution,  and  post-employment  duties  and 
responsibilities. 

5.  Review  AOD,  ATO  folder,  and  all  other  pertinent  documents. 

6.  Use  system  applications  to  accomplish  mission. 

7.  Provide  battle  damage,  operational,  and  process  assessments  when  available. 

8.  Request  ACO  ehanges  as  required. 
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9.  Monitor  availability  of  all  tasked  air  and  space  forces. 

10.  Suggest  changes  to  missions  of  the  air  and  space  operation,  which  must  be  coordinated 
through  various  agencies  inside  and  outside  the  AOC. 

11.  Coordinate  with  subordinate  units  of  the  TAGS  (especially  the  ASOC/tactical  air  control 
party  [TACP]  for  the  CAS/interdiction  cells). 

12.  Evaluate  the  degree  to  which  actual  operations  are  meeting  ATO  objectives. 

13.  Pass  on  critical  information  to/from  respective  WOCs  and  platforms  concerning  air  raid 
warnings,  significant  battle  damage,  unexpected  changes,  diverting  aircraft,  and  airfield 
status. 

The  Offensive  Operations  Team  reports  to  the  CCO. 

Members  are: 

•  Senior  Offensive  Duty  Officer  (SODO) 

•  SODO  Technician 

•  Offensive  Operations  Team  Members 

•  EW/SEAD  Duty  Officers 

•  Close  Air  Support  Duty  Officers 

•  Command  and  Control  Duty  Officers 

•  Interdiction  Duty  Officers 

•  Operations  Duty  Officers 

•  Tanker  Duty  Officers 

•  Airlift  Duty  Officers 

•  Dynamic  Targeting  Cell 

•  Operations  Duty  Technicians 

•  10  Duty  Officers/NCOs 

•  CSAR  Duty  Officers 

Defensive  Operations  Team  Tasks 

1.  Supervise  defensive  operations  during  eaeh  shift  with  speeial  emphasis  on  integrating  all 
defensive  and  support  operations. 

2.  Monitor  the  eurrent  defensive  air  and  theater  missile  defense  operations  and  advise  the  CCO 
of  dynamic  mission  requirements  and  resource  status. 

3.  Reeommend  immediate  ehanges  to  the  ATO  when  the  situation  dietates. 

4.  Assist  CCO  and  SADO  in  pre-employment,  execution,  and  post-employment  duties  and 
responsibilities. 

5.  Review  appropriate  documents  such  as  AOD,  ATO  Folder,  AGO,  and  ATO. 

6.  Conduct  briefings. 

7.  Coordinate  with  other  AOC  teams/eells  as  required. 

8.  Provide  inputs  to  BDA  and  other  assessment  as  required. 

9.  Provide  inputs  to  the  SITREP. 

The  Defensive  Operations  Team  reports  to  the  CCO. 

Members  are: 

•  Senior  Air  Duty  Officer  (SODO) 

•  SADO  Technician 

•  Defensive  Duty  Officers 
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•  Defensive  Duty  Technicians 

•  Command  and  Control  Duty  Officers 

•  Defensive  Counterair  Duty  Officers 

•  AWACS  Duty  Officers 

•  JSTARS  Duty  Officers 

•  Theater  Missile  Defense  (TMD) 

•  Theater  Missile  Defense  Officers 

•  Theater  Missile  Defense  Technician 

Team  Tasks 

Provide  situational  awareness,  threat  warning  and  identify/amplify  tracks/targets  in  support 
of  dynamic  targeting. 

Monitor  execution  of  the  current  day's  ATO  and  work  with  the  SODO's  Dynamic  Targeting 
Cell  to  provide  direct  support  (target  identification,  targeting  data,  weaponeering,  BDA,  etc) 
to  the  cod's  re-role,  dynamic,  and  TST  targeting  processes. 

Monitor  the  execution  of  the  ATO  and  RSTA  Annex  and  in  coordination  with  RDO/ISR 
Platform  LNOs  and  the  SIDO,  work  with  the  CCO,  SADO,  and  SODO  to  dynamically 
adjust  ISR  assets  and  Processing,  Exploitation,  and  Dissemination  (PED)  nodes  to  meet  the 
current  situation. 

Manages  the  ISR  assets  assigned  or  made  available  to  the  J/CFACC. 

Provides  real-time  exploitation/support  (IMINT,  MASINT,  ELINT,  and  COMINT)  to  the 
COD  during  current  ATO  execution. 

Coordinate  re- tasking  and  adjustment  of  PED  nodes  as  required  due  to  emerging  collection 
requirements  and  battlespace  changes. 

The  SIDO  Team  reports  to  the  CCO. 

Members  are: 

•  Senior  Intelligence  Duty  Officer  (SIDO) 

•  SIDO  Team 

•  Intelligence  Duty  Officers/Technicians 

•  Target  Duty  Officers/Technicians 

•  ISR  Operations  Duty  Officers/Technicians 

•  RDO/ISR  Platform  LNOs 

•  Multi-INT  Exploitation  Cell  (MEC) 

•  PED  LNOs 

Interface  Control  Team  Tasks 

1 .  Based  on  guidance  from  the  AADC  and  the  OPLAN,  determines  data  link  participants,  their 
equipment  capabilities  and  limitations  and  respective  needs. 

2.  Assists  C2  Plans  in  the  design  of  the  data  link  architecture  and  production  of  the  OPTASK 
LINK. 

3.  Ensures  optimum  Data  Link  connectivity  by  monitoring  the  Data  Link  and  directing  changes 
as  necessary. 

4.  Manages  the  data  link  network. 

5.  Provides  the  C/JFACC  with  a  consolidated  and  accurate  air  picture. 

6.  Provides  direction  to  attached  units  relative  to  alert  status. 


SIDO 

1. 

2. 

3. 

4. 

5. 

6. 
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7.  Produces  and  maintains  the  Reeognized  Air  Picture  (RAP)  by  managing  the  pietures 
produeed  by  subordinate  C2  units. 

8.  Coordinates  with  the  Operations  Common  Operational  Pieture  (COP)  Manager  for  the  air 
component  input  to  the  COP. 

9.  Builds  and  disseminates  the  multi-tactical  data  link  (TDL)  air  picture  used  to  support 
situational  awareness  for  eombat  operations  and  for  exehange  of  digital  messages. 

10.  Provides  multi-TDL  conneetivity  for  maehine-to-machine  interoperability  supporting  J- 
series  message  exchange  for  dynamic  targeting  and  prosecution  of  DT/TST  operations. 

1 1 .  Data  link  architecture  developmenCexecution. 

12.  GCCS  COP  development. 

13.  Provides  management  of  surveillance  operations. 

The  Interfaee  Control  Team  reports  to  the  CCO  or  the  SADO  (as  designated  by  the  CCO). 

Members  are: 

•  Interface  Control  Officer  (ICO) 

•  Link  11  AB  and  16  Managers 

•  Track  Data  Coordinators 

•  Track  Data  Technicians 
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Combat  Plans  Division  inputs  and  Outputs 


Combat  Operations  Division  Inputs  and  Outputs 


Consolidated  reports  to 
higher  headquarters 


Appendix  B.  Interviews  at  TRS 


Interview  1: 


Current  posn:  505TRS/DOC,  C2  Duty  Officer 

Background  includes:  AW  ACS,  Exercise  Plans  (Okinawa),  Contingency  Plans,  C2 
exercises,  16  AF  (AFFOR,  JFOR  staffs),  Curriculum,  ATO  visualization 

•  ATO  Mission  Planning 

o  Flow,  color,  callsign,  mission  number,  time-on-target  (TOT),  refueling  for  4- 
hour  glance  of  packages 
o  Filtered  up  to  what  TBMCS  has  nowadays 
o  “ESTAT  (Execution  Status)  is  great  if  it  works  correctly” 

•  Asa  C2DO  (or  any  posn  on  the  Combat  Ops  floor),  would  like  to  see:  Geography 
(airspace  control  measures),  ESTAT  picture,  comms  with  AOC 

•  “ADOCS  has  a  nice  “Find”  function 

•  Not  familiar  with  Targeting,  Weaponeering,  Intel 

•  C2PC,  Unix-side  map,  along  with  ADOCS,  for  SA 

•  TST  coordination  displayed  on  Data  Wall 

•  Geography: 

o  Combat  Air  Patrols  (CAPs),  air  refueling,  C2  orbits 
o  TST  CAPs  (tankers  and  air  assets  for  the  day) 
o  Target  locations,  red  force  locations 
o  Blue  air  bases 

•  “C2DO  needs  to  know  everything,  but  needs  to  be  able  to  filter  data” 

•  “(schoolhouse)  needs  to  teach  how  to  sort  and  filter” 

•  “love  ESTAT  because  it  looks  like  Excel;  you  can  eliminate  a  lot  of  duplicated 
info” 

o  Graphics  and  text 

•  Text  side:  mission  number,  callsign,  number  and  type  of  aircraft,  mission, 
package  ID, 

•  Graphical  side:  actual  time  of  departure,  on-station,  air  refueling  time,  TOT, 
color-coded  lines  for  status,  number  and  type  of  aircraft 

o  These  should  be  sorted  and  filtered  accordingly 

•  “Most  of  the  defensive  guys  know  how  to  use  the  filtering,  but  offensive  guys 
usually  do  not  since  there  are  so  many  of  them” 

•  C2  net 

•  IWS  (Info  Workspace):  uses  IWS  Chat  function 
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•  Full-up  TMBCS  in  the  schoolhouse  (w/  no  SIPRNet  feed) 

•  ESTAT  provides  an  ATO  visualization  that  is  adequate;  may  want  to  add  a 
feature  that  filters  out  text 

o  Has  not  used  ESTAT  for  a  large  number  of  aircraft 
o  ESTAT  is  part  of  TBMCS 

•  Joint  Defensive  Planning  Tool:  modeling  tool  for  Planning,  not  Ops 
Data  wall: 

•  TST  display  is  good 

•  “no  filtering”  seems  to  occur;  need  a  way  to  convert  feeds  (Data  link,  Blue  Force 
Tracker,  etc.)  into  a  layman’s  language 

o  Translator  for  symbology  is  important 

•  “End-of-course  exercise  is  not  real,  but  useless” 

•  Important  info  (for  data  wall)  is:  high-priority  taskings,  air  defense  warnings, 
army’s  ground  picture 


Interview  2: 

Current  posn:  Senior  C2  Analyst,  Ops  Manager 

Background  includes:  CENTAF,  A3,  A5,  AOC  Director,  Weasels,  F-4s,  A- 10s,  AOC  C2, 
AOC  TTP 

Data  Wall: 

As  the  AOC  Director,  there  are  certain  things  I  need  to  see: 

•  “COP  is  useful  only  when  measured  against  the  plan” 

•  ICC  (NATO  TBMCS)  -  everyone  in  NATO  uses  ICC,  except  for  the  U.S. 

o  COP  (AOR),  aircraft,  callsign,  allows  a  comparison  of  COP  and  planned 
ATO 

o  COP  is  probably  not  looked  at  very  much 
o  Airspace  deconfliction  is  important 
o  ESTAT  does  well  with  flow 

•  Most  tools  do  not  teach  you  how  to  do  a  good  job;  they  don’t  teach  you  how  to 
better  do  what  you  do 

•  COP  must  be  “zoomable”  into  area  of  interest  (CAS,  TST,  CSAR,  etc.) 

•  In-flight  emergencies 

•  Blue  Force  Tracker 

•  Part  Task  Trainer  (PTT)) 

o  ICC  training  capability 

o  2-terminal  thing,  build  ATOs,  modeling  and  planning,  fly  ATOs  over  and 
over,  shoot  down  bad  guys 

•  24-hour  flow — ^personal  preference 
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•  Looking  at  flow  should  allow  you  to  see  any  gaps  (airspace  deconfliction) 

•  Filtering  allows  you  to  see  defensive  and  offensive 

•  Predator,  U-2,  CAP,  air  defense,  protection  for  strike  packages 

•  Tool  will  help  planners  as  well 

•  Tool  should  display  change  to  the  ATO 

•  As  AOC  Director,  used  the  data  wall  throughout  the  day  or  when  someone 
brought  something  to  attention 

o  “Would  probably  benefit  everyone  to  see  their  phase  (role)  in  the  overall 
plan.” 

o  Trails  of  aircraft  (breadcrumbs) 

ATO  Visualization 

•  Air  refueling  tanker  planning  is  not  connected  to  MAAP  Toolkit  but  probably 
should  be 

•  ATO  visualization  tool  might  graphically  display  when  the  aircraft  is  going  to 
run  out  of  gas  and  whether  or  not  the  pilot  will  be  able  to  complete  the  mission 

•  Filter  capability — just  SEAD  or  DC  A  flow 

•  Considers  ESTAT  a  good  ATO  visualization  tool 

•  Most  folks  go  back  to  using  a  whiteboard  from  the  computer  to  do  the  ATO 

•  Collaboration  is  important 

•  Potential  feature:  drag  and  drop  of  routes  should  be  a  potential  feature 

•  Displaying  assets’  capabilities,  ROE,  limitations,  coalition  nation  and  host  nation 
assets 


Interview  3: 

Current  posn:  505  TRS/DOC  Flight  Commander 

Background  includes:  B1  Weapons  System  Officer,  DESERT  FOX,  Mission  Planning 
Cell 


•  ESTAT  is  primary  tool;  many  varied  applications  on  TBMCS 

•  Primary  concern:  is  the  tool  manageable?  -  3,000  sorties  is  a  lot,  150-200  is 
manageable 

•  “ESTAT  is  Excel  from  hell” 

•  Striker  perspective:  targets,  threats,  refuelers,  status  of  strike  packages,  status  of 
strikes — destroyed? 

•  COP  doesn’t  necessarily  display  targets 

•  PTT  displays  ATO  as  it’s  being  played  out  through  TBMCS;  not  a  3D  view,  but 
it  should  be 

•  Prioritized  targets  list — on  screen 

•  Refueler  plan  is  key;  deconfliction  is  key 

•  “F-1 17  is  going  away  soon” 
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1400  -  Lt  Col 


Current  posn:  705  TRS/DO 

Background  includes:  C2WAC,  strat  div  planner,  JSSE,  CSSE,  AOC  Director,  chief  of 
plans,  chief  of  ops,  OTCP  (tier  3),  Cell  and  team  chiefs,  TST,  Combat  ops,  MAAP  and 
combat  ops,  instructor 

Data  Wall: 

•  Automated  stuff  -  doesn’t  use  it 

•  ADSI  feed  -  will  look  at  it;  more  real-time 

•  MAAP  Brief,  bits  of  info 

•  “ADOCS  is  good  enough” 

•  If  I  had  my  choice,  Ed  want  to  have  a  link- 16  picture  as  a  COP 

•  DEARS — tested  at  last  JEFX,  “ESTAT  tool  for  TBONE”,  real-time  data  (tied  to 
link- 16) 

•  Seeing  threats  as  they’re  populated 

•  Rover  feeds,  predator  feeds 

•  Data  wall  might  include:  Link- 16  COP  (or  ADSI)  or  some  other  real-time  feed; 
CAS  stuff;  BCD;  ROE,  Guidance;  JIPTL,  MAAP 

•  ESTAT  should  contain  the  most  updated  info 

•  ADOCS  (Army’s  system)  shows  what’s  planned 

•  Tailor  ESTAT  to  log-in,  remember  preferences  whenever  ADOCS  does  not 

•  Speech  recognition  would  be  a  great  feature;  operators  already  have  headsets  and 
microphones  on  at  their  positions 

•  My  perfect  tool: 

o  ADOCS-based;  ADOCS  right-click  to  find  on  a  map 
o  DEARS  +  ESTAT  -I-  Joint  Targeting  Toolkit 
o  Point  and  click,  right  click 
o  COP  driven  by  Link  16 

•  Existing  apps  do  a  fairly  good  job  of  ATO  visualization 

•  Priority  tools  I  use:  ADOCS,  ESTAT,  IWS  for  collaboration 


Interview  4: 

Current  posn:  505  TRS/DO 

Background  includes:  Air  Battle  Manager  for  AWACS;  C2  Planner;OIF,  SADO  on 
Combat  Ops 

•  Consolidation  of  apps  would  be  awesome 
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MAAP  Toolkit:  great  for  2,000  sorties,  but  not  fewer 
Web-based  functions  work  best 


From  a  defensive  perspective: 

MIRC  chatting  collaboration  with  SADO,  SICO,  Air  Defnse  Warning  chanes, 

AAMD  &  others,  folks  within  the  AOC, 

Rotisserie:  status  of  C2  facilities;  weapons  contrl  status;  airbase  status,  C2  status. 
Weather,  space  weather. 

Four  main  screens:  1)  COP,  2)  threat  picture,  3)  ADOCS  picture,  4)  CNN  (open 
source  news) 

lOF :  spent  much  time  coordinating 

ADOCS,  SAA,  C2PC:  data  flows  differently  between  these  3;  ADSI  was  most 
accurate 

Combine  or  separate  air  and  ground  picture;  customizable;  same  functionality  as 
ADOCS;  overlay  of  ADSI  picture;  capability  for  rotisserie  (web-based  TBMCS  data); 
automated  weather  feed 
Data  back-up  is  critical 

In-flight  status,  aircraft  status,  etc.  did  not  show  up  at  times 
o  Used  ESTAT  a  lot 
o  Mutli-layer  sorting  helped  a  lot 
o  Power  point  print  out  of  ISR  assets  &  collection  desk 
IWS  chat  rooms  multiple  rooms 
Real-time  updates  would  make  ESTAT  better 

o  Marrying  ESTAT  and  JTIDS  would  be  great 


Interview  5: 

Current  posn:  Offensive  curriculum  development;  instructor 

Background  includes:  C-130  pilot;  T-37  trainer;  misawa  AOC,  Wing  weapons  officer;  12 
AF;  CTAPS,  airlift.  Combat  Plans;  Combat  Ops,  Chief  of  ATO  production;  SOUTAF 
Ops  Cell,  stood  up  AOC  at  Hawaii  (PACAF) 


•  Trying  to  reduce  the  number  of  people  for  an  AOC  is  not  going  to  work 

o  Experts  are  needed  for  each  function 
o  Number  of  people  is  tied  to  the  number  of  functions 
o  Skill-set  based 

•  Data  walls  are  useless:  CNN  and  football 

o  Big  screens  are  created  for  the  JFACC 
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o  Four  screens:  1)  CNN  or  FOX  (open  source  news);  2)  COP  or  tactical  air  picture 
(near-real  time)  and  ground  foree  pieee;  3)  Rotisserie:  AOC  Direetor  guidance, 
weather  and  negative  impaet  on  missions,  admin;  4)  “wild  card”  screen:  ESTAT 
(Execution  STATUS);  FSTAT  (Forces  Status) 

ATO  Visualization: 

•  Simplified  version  of  PTT  might  be  a  good  ATO  viz  tool 

•  Grabbing  AFMIS  tapes  and  auto  forwarding  to  a  display  or  to  Chief  of  Combat  Ops 

•  “I  think  ATO  should  feed  AFMIS” 

•  Unit  level  to  Foree  level  TBMCS;  still  do  paper  copies  of  ATO 

Note:  level  1,  2,  and  3  BDA  have  changed;  Joint  Pub  governing  Combat  Assessment  and 
BDA  levels 

•  “ESTAT  is  an  ATO  viz  tool” 

•  “ESTAT  is  filterable” 

o  Should  be  able  to  view  all  airspace  or  a  selectable 

•  “ESTAT  is  a  tabular  representation  of  the  ATO 

o  3D  graphic  with  time  lapse 

o  Most  sorties  ever  seen:  4,700,  a  dozen  at  a  time,  200  in  a  shift 

Interview  6: 

Current  posn:  DOI;  Intel/ISR  DO  trainer,  specialize  in  targeting. 

Baekground  ineludes:  14N,  JEFX,  C2  ISRC,  ISR  Manager,  Wing  IN,  Imagery 
Data  Wall: 

•  Useful,  exeept  for  what  might  be  distraeting  sueh  as  Predator 

•  Data  Walls  should  be  added  inside  the  SCIF 

o  COP  is  critieal  (red  and  blue);  used  as  a  prompt  to  eall  between  ops  floor  &  Intel 
■  Need  to  integrate  the  ISR  COP;  eurrent  version  presents  web-based 
viewing  (ISR  Warrior) 

Note:  AOC  in  Korea  should  be  considered  for  data  eolleetion. 

•  On-deek  for  ATO  (flow) 

•  Weather  picture 

•  In-flight  report,  BDA,  mission  reports  time  sequential  list  with  color  codes  and  pie 
chart  colored  as  the  ATO  progresses  (entire  ATO),  related  objeetive,  airbase  status 

•  Rotisserie 

o  CNN  &  FOX  news 
o  Centrally  visible 
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o  IWS  chart  is  used  mostly,  then  MIRC  chat;  IWS  craps  out  a  lot  more  than  MIRC, 
but,  provides  more  capability  than  MIRC 

•  Combine  ISR  and  flying  side 

•  Color  eode  paekages  and  asset  types 

•  Setting  defaults  is  painful 

•  Combine  paekages  with  reporting 

•  Highlight  packages  that  get  re-rolled 

•  Weather  and  space  effeets  needs  to  be  there  (overlays,  transparencies) 

•  Positive  ID  (eorroborating  sourees) 

•  Display  of  assets  available  for  re-roll  (re-task)  (more  than  just  aeft) 


Interview  7: 

Current  posn:  Strategy  Course  Director 

Background  includes:  Air  Battle  Manager  experienee;  AW  ACS,  TTP  for  AF,  JEFX  TTP 
(TST);  Vieenza,  Prinee  Sultan  Air  Base  (PSAB),  Korea,  AOC  experienee.  Combat  Plans, 
Combat  Ops,  Strategy 

•  JEFX  2000  -  Data  Wall 

•  Management  of  info  sharing  and  data 

•  Predecessor  to  TST  tools 

•  Resident  ISR  expert:  Mike  England 

•  Combat  Ops:  execution  visualization  tool 

•  Combat  Ops:  Ops  assessment — a  viz  tool  would  be  beneficial 

•  Combat  Plans — MAAP  Toolkit — resident  expert  on  MAAP  Toolkit  is  “Coaeh”. 

•  Air  picture 

•  Info  sharing 

•  General  skeds 

•  ADOCS  -  SA  Tool 

•  Data  management 

•  Info  sharing 

•  Decision-making  support 

Pepe’s  Suggested  Reading  List: 

o  AOCCONOPS 

o  Joint  Pubs  eurrently  in  development 
o  AFOTTP  2.3.2 
o  Training  task  list 


141 


Interview  8: 


Current  posn: 

Background  includes:  C-130  pilot;  8  AF;  C2WAC,  PSAB,  Internal  Look,  OIF,  OSW, 
OCTP,  ATO  production,  Plans  Guy 

Note:  6-12  May:  several  personnel  from  505  TRS  are  traveling  to  A1  Udeid  to  determine 
where  the  AOC  there  is  headed. 

•  Does  not  use  ESTAT,  since  ESTAT  is  an  execution  tool,  not  a  planning  tool 

•  “flow  sheef  ’  would  help  in  the  ATO  production  phase 

•  Wait  for  MAAP  to  be  completed  in  MAAP  Toolkit  and  entered  into  TBMCS; 
tankers;  technical  accuracy  is  critical;  tankers  are  done;  send  to  ATO  wings;  sent  to 
Ops  Floor 

•  TBMCS  doesn’t  catch  a  lot  of  common  errors 

•  Ops  floor  operators  pump  it  into  the  system 

•  MAAP  is  used  to  fat-finger  it  in 

•  Technical  accuracy  is  checked  by  tech 

•  MAAP  Chief,  ATO  production 

•  ATO  vis  would  be  beneficial  in  Plans 

•  Flyout  tools  hasn’t  seen  them 

•  Rainbow  sheet 

•  Ops  guy  don’t  have  means  of  talking  to  Combat  Plans 

•  ATO  Coordinator:  supposed  to  accompany  the  ATO;  LtCol  type 

•  Filters,  sorting 

•  Flyouts  not  necessary  for  ATO  Plans 


Interview  9: 

Current  posn:  CSAR  C2  analyst 

Background  includes:  HH-60  pilot;  Special  Ops  Liaison  Element  (SOLE),  test  &  eval, 
AOC  Duty  Officer, 

•  “ESTAT  is  not  necessarily  a  visualization  tool’’ 

•  Visualization  tool  should  be  a  template,  customizable 

•  Current  ADOCS  display  is  good  for  Rescue 

•  Data  Wall:  should  include  display  of  routes  and  rescue  choppers  as  well  as  CSAR 
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Appendix  C.  Visualization  Prototypes 


File  Filters  Sort  By 

Search:  I  Results  View  ford 


nci^ 


Data  Tag  Mode:  I  Default 


JB 


B  Unnamed 


Mission  Type  0600  |  0700  |  0800  |  0900  |  1000  |^U00  1 1200  1 1300  1 1400  |  1500  1 1600  |  17^ 


j  Package 


Preview  |  Layers  |  Target  Filter  | 

n  Fire  Support  Coordination  Lin  (FSCL) 
r  Forward  Edge  of  Battle  Area  (FEBA) 
r*  Forward  Lin  of  Own  Troops  (FLOT) 
I”  Restricted  Operation  Zone  (ROZ) 
r  Combat  Air  Patrol  (CAP) 
r  Safe  Passage  Corridors 
r  Kill  Box  Grid 
r  Air  Refueling  Tracks 
r  ISR  Orbits/Tracks 
r"  No  Fly/Restricted  Fly  Areas 
r  Prohibited  Areas 
F  Mine  Areas 


B  2A00 

1 

C  1  Q"3QQ77 

1  A  10007// 

•  13/ob34 
^  1 393334 

B  ^  2B01 

ATTACK 

&  1578564 

1 

A  1 070007 

&  1954434 

SEAD 

;i 

< 

> 

Weapons 


Overview  -  This  shows  the  view  of  a  single  target,  or  DMPI.  Holding  the  cursor  over  the 
target  will  reveal  information  on  the  target,  including  but  not  limited  to:  BE  number, 
target  type,  weapon  selected,  and  delivery  method.  The  focus  of  this  view  is  focused  on  a 
single  target  and  purposefully  leads  to  slides  that  focus  on  the  big  picture  and  how  the 
user  arrives  at  assigning  weapons  and  packages  (a  package  being  a  group  of  aircraft  that 
work  in  tight  cooperation  to  achieve  a  focused  goal)  to  a  specific  target. 

Functionality  -  Timeline  View  on  the  top  of  the  screen  shows  the  user  the  overall 
timeline  for  planning  purposes  including  available  aircraft  packages,  weather  information 
and  sunrise/sunset  and  moonrise/moonset  information.  Vertical  timeline  represents  the 
point  in  time  when  this  target  is  being  engaged. 

Assignment  Windows  -  Windows  on  the  right  side  give  user  the  ability  to  drag-and-drop 
icons  onto  target.  If  “target  type”  is  not  available  from  BE  data  or  if  the  user  wants  to 
change  the  target  type  they  can  drag-and-drop  a  new  designation  onto  the  target.  User 
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can  reference  JMEMS  or  drag  target  onto  JMEMS  interfaee  for  reeommended  weapons. 
Weapons  ean  be  dropped  onto  targets  on  the  map.  Weapons  are  linked  with  aireraft  that 
have  the  eapability  to  deliver  them.  After  a  weapon  is  seleeted  only  appropriate  aircraft 
will  be  highlighted,  the  rest  will  be  grayed  out.  Selected  target  is  “linked”  visually  with 
Package  in  the  Timeline  view. 

Tabbed  Windows  -  These  tabs  give  the  user  a  great  deal  of  flexibility  when  viewing  the 
map,  when  seleeting  data  to  be  represented  on  the  map,  when  searehing  for  target 
information  and  when  operating  in  “simulation  mode”.  Options  can  be  selected  at  any 
time  by  the  planner,  providing  a  wealth  of  easily  understood  information. 

Transition  to  Slide  2  -  It’s  critieal  to  acknowledge  that  a  single  target  doesn’t  exist  in 
isolation.  Any  target  is  part  of  a  mueh  larger  plan  and  needs  to  be  eonsidered  in  that 
eontext.  In  context,  this  target  is  one  target  of  four  assigned  to  one  attaek  aircraft,  the 
attack  aircraft  is  part  of  a  package  ineluding  support  aircraft  of  varying  types,  that 
package  has  to  work  in  eoneert  with  other  paekages  whieh,  in  turn,  are  coordinated  with 
still  more  paekages. 
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File  Filters  Sort  By 

:  ^  tt  -  Search:  [Results  View  forP  If^^l  •  Data  T aq  Mode:  I  Default  I  t] 


f”  Fire  Support  Coordination  Lin  (FSCL) 
r  Forward  Edge  of  Battle  Area  (FEBA) 
r*  Forward  Lin  of  Own  Troops  (FLOT) 
r*"  Restricted  Operation  Zone  (ROZ) 
r  Combat  Air  Patrol  (CAP) 
r"  Safe  Passage  Corridors 
r  Kill  Box  Grid 
P  Air  Refueling  Tracks 
I”  ISR  Orbits^racks 
P  No  Fly^estricted  Fly  Areas 
P  Prohibited  Areas 
P  Mine  Areas 


This  screen  is  much  earlier  in  the  planning  proeess.  The  planner  sees  all  targets  in  the 
vieinity  (as  opposed  to  a  single  target)  and  needs  to  assign  weapons  and  aireraft  packages 
to  eaeh.  The  planner  will  need  to  take  into  aceount  a  multitude  of  factors,  some 
represented  in  the  “Layers”  tab  to  the  left  of  the  map,  but  also  ineluding  available 
armaments,  weather,  time  of  day,  defenses  and  available  packages. 


The  Timeline  view,  at  this  point,  represents  what  paekages  are  available  at  what  times. 
The  planner,  factoring  in  all  pertinent  information  will  seleet  the  appropriate  paekages 
and  assign  them  to  a  group  of  targets.  Then  this  “group”  will  be  part  of  the  overall  ATO. 


The  following  sereenshots  show  how  this  part  of  the  ATO  will  play  out  based  on  the 
planners  eoordination  of  resources. 
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This  screenshot  represents  the  beginning  of  planning  for  a  specifie  paekage,  2B00.  The 
timeline  is  set  to  represent  the  departure  of  the  packages  attack  aircraft.  At  this  point  the 
user  has  some  overall  idea  of  what  types  of  targets  he  needs  to  attaek,  what  aircraft  are 
available,  what  weaponry  is  available  an  any  other  constraints  on  his  work.  With  all 
input  considered  he  knows  that  this  particular  package  “matehed  up”  with  this  set  of 
targets.  So  he  assigns  the  package  and  provides  ingress/egress  direetions,  approximate 
times  and  moves  to  the  next  set  of  targets. 
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File  Filters  Sort  By 
•  ^  It  :  Search:  [Results  View 


Efort 


ZIChD  •  Data  Tag  Mode:  I  PetaulT" 


□  ResultsView  £3 


Package 

Mission  Type 

0600  1  0700  1  0800  |  0900  |  1000  I'^llOO  |  1200  |  1300  |  1400  |  1500  | 

IBBBBSI!!  1800  1900  2000 . 2100  ;  220i| 

□•■*9  2A00 

:  £  1838977 
;  &  1373634 

TANKER 

ATTACK 

•  £  1393334 
2B00 

_  C  1  o-^OQ77 

C2 

AWACS 

W  IDJD?/ / 

-  £  1  ■^7QtQd. 

ATTACK 

..  .  £  1 

d  dgi  2B01 

ATTACK 

“ 

TftML^FD 

. £  1578564 

ATTACK 

•  ijyoJjT 

•  £  1954434 

5EAD 

 ^1 

Preview  |  Layers  [  Target  Fitter  ] 

r*  Fire  Support  Coordination  Lin  (FSCL) 
r  Forward  Edge  of  Battle  Area  (FEBA) 
r  Forward  Lin  of  Own  Troops  (FLOT) 
Restricted  Operation  Zone  (ROZ) 
n  Combat  Air  Patrol  (CAP) 
f  Safe  Passage  Corridors 
r  Kill  Box  Grid 
P  Air  Refueling  Tracks 
P  ISR  OrbitsTTracks 
P  No  Fly>Restricted  Fly  Areas 
P  Prohibited  Areas 
P  Mine  Areas 


0  Unnamed  E3 


Other  Aircraft  ^  g 


This  shows  the  final  plan  for  this  set  of  targets.  Three  packages  will  attack  the  targets. 
Selecting  an  arrow  will  provide  the  details  associated  with  each  package  and  indicate 
which  targets  are  being  attacked,  approximate  times  of  attack,  aircraft  in  the  package,  and 
armaments  being  used. 


147 


File  Filters  Sort  By 

:  4  n  ;  Search: [^sults  View  l^\  forC 


■  Data  Tag  Mode:  I  DefauiP 


JB 


(2  Unnamed  £3 


Package 

Mission  Type 

0  ®  2A00 

;  S  1838977 

TANKER 

£  1373634 

ATTACK 

£  1393334 

C2 

0  ®  2800 

£  1838977 

AWAC5 

£  1373634 

ATTACK 

£  1393334 

ATTACK 

0  ®  2E01 

:  £  1578564 

TANKER 

;  £  1398334 

ATTACK 

£  1954434 

SEAD 

0600  I  0700  I  0800  |  0900  |  1000  | 


M= 


Visualization  X 


Preview  ]  Layers  |  Target  Filter  ] 

l~  Fire  Support  Coordination  Lin  (FSCL) 
P  Forward  Edge  of  Battle  Area  (FEBA) 
P  Forward  Lin  of  Own  Troops  (FLOT) 
P  Restricted  Operation  Zone  (ROZ) 

P  Combat  Air  Patrol  (CAP) 

P  Safe  Passage  Corridors 
P  Kill  Box  Grid 
P  Air  Refueling  Tracks 
P  ISR  Orbits/Tracks 
P  No  Fly/Restricted  Fly  Areas 
P  Prohibited  Areas 
P  Mine  Areas 


n  ResultsView  £3 


-J, 

Weapons 

Package  Aircraft 


Support  Aircraft 


Other  Aircraft 


This  screenshot,  and  the  next  two,  act  as  a  sort  of  simulation  of  the  ATO  Plan  in  action.  The  time 
bar  in  the  Timeline  View  will  be  moved  across  the  screen  and  aircraft  packages  will  attack  the 
targets  on  the  map.  As  the  time  bar  follows  the  map,  the  arrows  in  the  map  view  appear  and 
disappear  as  the  attack  aircrafts  are  scheduled  to  hit  their  targets.  The  arrows  are  keyed  to  attack 
aircraft  because  other  aircraft  may  be  in  support  of  other  packages  as  well.  This  clarifies  for  the 
user  when  attack  missions  are  occurring  and  will  also  inform,  in  a  generic  manner  via  the  map  -  a 
specific  manner  via  the  Timeline,  that  other  support  aircraft  in  the  area.  With  the  entire  ATO 
planned  we  know  there  are  3  “attacks”  that  will  handle  the  9  targets  on  the  map. 

The  first  attack.  Package  2A00,  follows  the  path  of  the  arrow  and  strikes  the  3  green  targets  with 
the  universal  red  “no”  symbol  superimposed  on  them.  In  subsequent  screens  the  same  coding 
will  be  used  to  indicate  targets  that  should  have  already  been  attacked.  The  red  dots  indicate  the 
targets  for  our  selected  package  (note:  package  2B00  should  be  highlighted  in  the  Timeline 
View).  Green  dots  indicate  targets  for  non-selected  packages.  Icons  on  particular  aircraft’s 
timeline  indicate  time  of  attack. 
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At  this  point,  we  can  see  that  the  first  attack  package  is  still  in  the  air  after  attacking  the 
targets  and  the  second  attack  is  yet  to  occur.  The  presence  of  the  arrows  on  the  screen 
indicates  that  the  aircraft  are  in  the  air,  by  looking  at  the  timeline  view  we  see  the  status 
of  the  attacks. 
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File  Filters  Sort  By 

;  ^  It  ;  Search: [^sults  View  for:|Z 


1  Data  Tag  Mode:  I  PefauiT" 


[2  Unnamed  £3 


Package 
□  2A00 

;  &  1838977  TANKER 
;  &  1373634  ATTACK 

•  &  1393334  C2 
2B00 

:•&  1838977  AWACS 
'  1373634  ATTACK 

&  1393334  ATTACK 
d  -#  2B01 

;  &  1578564  TANKER 
;  &  1398334  ATTACK 

•  &  1954434  SEAD 


Mission  Type  0600  |  0700  |  0800  |  0900  |  1000  |  V^OO  )  1200  |  1300  |  1400  |  1500  |  1600 1  1700 


1800  I  1900  I  2000  I  2100  I  2?0i 


Preview  ]  Layers  |  Target  Fitter  ] 


r  Fire  Support  Coordination  Lin  (FSCL) 
r  Forward  Edge  of  Battle  Area  (FEBA) 
I”  Forward  Lin  of  Own  Troops  (FLOT) 
r"  Restricted  Operation  Zone  (ROZ) 

I”  Combat  Air  Patrol  (CAP) 
n  Safe  Passage  Corridors 
r  Kill  Box  Grid 
I”  Air  Refueling  Tracks 
r"  ISR  OrbitsTTracks 
n  No  Fly^estricted  Fly  Areas 
P  Prohibited  Areas 
P  Mine  Areas 


□  ResultsView  E3  □ 


Weapons 

Package  Aircraft 


Support  Aircraft 

Other  Aircraft 

The  final  slide  of  the  “simulation”,  similarly  to  the  seeond,  shows  the  status  of  the  aireraft 
and  targets  at  a  glanee.  We  ean  immediately  see  that  the  first  attaek  has  occurred  and  that 
the  attack  aircraft  are  out  of  the  area,  the  second  attack  is  about  to  occur  and  the  final 
attack  is  now  in  the  air  and  approaching  the  target. 
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The  following  are  functionality  eoncepts  showing  various  tools  and  features  eould  be 
ineluded  to  aid  the  user  in  planning  an  ATO.  These  coneepts  are  presented  in  no 
partieular  order,  rather  just  as  a  set  of  ideas  that  can  be  incorporated  within  the  overall  UI 
design. 


Target  Filter  Tab.  Target  Filter  -  Allows  the  user  to  filter  what  targets  are  shown  in  the 
visible  map  area. 
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Prohibited  Areas  Layer 
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Forward  Line  of  Own  Troops 
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Mine  Areas 
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Sunset/sunrise  scroll 
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Weather  scroll  -  note  the  yellow  bars  indicating  length  of  weather  warning. 
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Vertical  sorter  -  The  map  allows  the  user  to  switch  from  a  bird’s  eye  view  to  a  horizontal 
view. 
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Preview  Tab.  Preview  Tab  -  Allows  the  user  to  move  around  in  the  map  area,  zoom-in, 
zoom-out,  etc. 


Acronym  List 

AADC 

Area  Air  Defense  Commander 

ABP 

Air  Battle  Plan 

ACM 

Air  Control  Measures 

ACP 

Airborne  Command  Post 

ADOCS 

Automated  Deep  Operations  Coordination  System 

ADP 

Air  Defense  Plan 

ADSI 

Air  Defense  Systems  Integrator 

AFB 

Air  Force  Base 

AFFOR 

Air  Force  Forces 

AFFORTTP 

Air  Force  Forces  Tactics  Techniques  and  Procedures 

AFMIS 

Air  Force  Management  Information  System 

AOC 

Air  &  Space  Operations  Center 
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AOD 

Air  Operations  Directive 

AOR 

Area  Of  Responsibility 

ASOC 

Air  Support  Operations  Center 

ATDM 

Advanced  Team  Decision  Making 

ATO 

Air  Tasking  Order 

AWACS 

Airborne  Warning  And  Control  System 

BDA 

Battle  Damage  Assessment 

BDC 

Battle  Damage  Control 

BE 

Basic  Encyclopedia 

C/JFACC 

Combined/Joint  Forces  Air  Component  Command 

C/JFC’s 

Combined/Joint  Force  Commander's 

C2 

Command  and  Control 

C2PC 

Command  and  Control  Personal  Computer 

C2WAC 

Command  &  Control  Warrior  Advanced  Course 

CAP 

Combat  Air  Patrol 

CCD 

Camouflage,  Concealment,  &  Deception 

CCO 

Chief  of  Combat  Operations 

CDM 

Critical  Decision  Method 

CENTAF 

U.S.  Central  Command  Air  Forces 

CNN 

Cable  News  Network 

COD 

Combat  Operations  Division 

COMAFFOR 

Commander  Air  Force  Forces 

COMINT 

Communications  Intelligence 

COP 

Common  Operational  Picture 

CPD 

Combat  Plans  Division 

CSAR 

Combat  Search  And  Rescue 

CSSE 

Combat  Service  Support  Element 

CTA 

Cognitive  Task  Analysis 

DCA 

Defensive  Counter  Air 

DEARS 

Data  Link  Automated  Reporting  System 

DMPI 

Desired  Mean  Point  of  Impact 

DOI 

Defensive  Operations  Instructor 

DT/TST 

Dynamic  Target/Time  Sensitive  Target 

ELINT 

Electronic  Intelligence 

ESTAT 

European  Surface-to-Air  Tactics  Analysis  Team 

EW 

Electronic  Warfare 

ICC 

Intelligence  Coordination  Center 

ICO 

Interface  Control  Officer 

ID 

Identification 

IMINT 

Imagery  Intelligence 

IRs 

Intelligence  Requirements 

IWS 

Integrated  Warfare  Systems 

IWS 

Integrated  Warfare  Systems 

JAOP 

Joint  Air  Operations  Plan 

JCEOI 

Joint  Communications-Electronics  Operations  Instructions 

JFOR 

Joint  Forces 
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JIPTL 

Joint  Integrated  Prioritized  Target  List 

JMEMS 

Joint  Munitions  Effectiveness  Manuals 

JRFL 

Joint  Restricted  Frequency  List 

JSSE 

Joint  Service  Support  Element 

JTC 

Joint  Theater  Commander 

JTL 

Joint  Task  List 

LINK 

Logistics  Information  Network 

MAAP 

Master  Air  Attack  Plan 

MASINT 

Measurement  and  Signatures  Intelligence 

MEC 

Mission  Essential  Competencies 

METOC 

Meteorological  and  Oceanographic 

MIRC 

Military  Intelligence  Reserve  Command 

NATO 

North  Atlantic  Treaty  Organization 

NCOIC 

Non-Commissioned  Officer  In  Charge 

NSL 

No  Strike  List 

OCTP 

Organizational  Command  Training  Program 

OIF 

Operation  Iraqi  Freedom 

OPLAN 

Operational/Operations  Plan 

OPTASK 

Operational  Tasking 

OSW 

Operation  Southern  Watch 

OTCP 

Officer  Training  Command  Pensacola 

PACAF 

Pacific  Command  Air  Forces 

PED 

Processing,  Exploitation  and  Dissemination 

PSAB 

Prince  Sultan  Air  Base 

PTT 

Part  Task  Trainer 

RAP 

Recognized  Air  Picture 

ROE 

Rules  Of  Engagement 

ROW 

Rest  of  World 

RPD 

Recognition-Primed  Decision  Model 

RSTA 

Reconnaissance,  Surveillance,  and  Target  Acquisition 

RTL 

Restricted  Target  List 

SA 

Situation  Awareness 

SAA 

Situation  Awareness  and  Assessment 

SADO 

Senior  Air  Defense  Officer 

SCIF 

Sensitive  Compartmented  Information  Facility 

SEAD 

Suppression  of  Enemy  Air  Defense 

SICO 

Sector  [or  Senior]  Interface  Control  Officer 

SITREP 

Situation  Report 

SME 

Subject-Matter  Expert 

SODO 

Senior  Operations  Duty  Officer 

SOLE 

Special  Operations  Liaison  Element 

SOUTHAF 

Southern  Command  Air  Forces 

SPINS 

Special  Instructions 

SRA 

SRA  International,  Inc. 

STO 

Special  Technical  Operations 

TACOPDAT 

Tactical  Operation  Data 
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TAGS 

Tactical  Air  Control  System 

TBMCS 

Theater  Battle  Management  Core  Systems 

TBONE 

Theater  Battle  Operations  Net-Centric  Environment 

TDL 

Tactical  Data  Link 

TET 

Targeting  Effects  Team 

TKA 

Team  Knowledge  Audit 

TMD 

Theater  Missile  Defense 

TNL 

Target  Nomination  List 

TOT 

Time  on  Target 

TST 

Time-Sensitive  Target 

TTP 

Tactics,  Techniques  &  Procedures 

WAW 

Warfighter  Analysis  WorkshopAFB  Air  Force  Base 
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